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PREFACE 


The  military  services  have  been  experiencing  problems 
in  the  adequacy  of  their  weapon  systems'  diagnostic 
capabilities.  Military  standards  and  handbooks  are  an  im¬ 
portant  factor  in  acquiring  these  diagnostic  capabilities. 
The  structure  and  content  of  existing  standards  and  hand¬ 
books  are  not  adequately  supporting  this  process.  The 
report  identifies  the  necessary  modifications  to  existing 
military  standards  and  handbooks  which  are  required  to  cor¬ 
rect  these  deficiencies. 
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INTRODUCTION. 


I . 

The  primary  objective  of  the  Testability/Diagnostic  En¬ 
cyclopedia  Program  is  to  develop  the  required  products, 
capabilities,  data,  and  guidance  for  the  cost-effective  and 
coordinated  specification  and  treatment  of  equipment  and 
system  testability/diagnostics  through  every  phase  of  the 
Acquisition  Cycle.  The  first  portion  (Part  I)  of  this 
program,  covered  by  contract  (F30602-86-C-0096) ,  is  aimed  at 
ascertaining  the  adequacy  of  existing  military  standards  and 
handbooks  in  dealing  with  the  application  of  required  tes¬ 
tability  and  diagnostic  principles  and  techniques  utilized 
in  the  acquisition  of  weapon  systems  and  providing  specific 
improvements  in  these  standards  and  guides,  as  required. 
The  specific  objectives  of  this  effort  (Part  I)  are  to: 

(a)  Review  existing  military  standards  and  handbooks 
in  the  fields  of  engineering,  reliability,  main¬ 
tainability,  logistics,  and  associated  hardware, 
software,  documentation,  and  training  with  respect 
to  essential  testability/diagnostic  coverage  and 
provide  draft  revisions  or  modifications,  as 
necessary,  to  provide  that  coverage/cons idera t i on ; 
and , 

(b)  Identify  those  diagnostic/testability  areas  which 
require  the  development  of  additional  military 
standards  and  handbooks  and  provide  rough  drafts 
of  their  content. 

As  depicted  in  Figure  1-1,  the  technical  approach  for 
achieving  these  objectives  is  divided  into  three  phases. 
Phase  I  deals  with  the  identification  of  existing  and 
planned  standards  and  handbooks  and  the  determination  of  re¬ 
quirements  for  standards  and  handbooks.  Phase  II  deals  with 
the  comparison  of  the  requirements  for  standards  and  hand¬ 
books  with  the  contents  of  existing  and  planned  standards 
and  handbooks.  The  output  of  this  phase  is  the  identifica¬ 
tion  and  scoping  of  changes  to  existing  standards  and  hand¬ 
books  and  the  identification  of  the  need  for  new  standards 
and  handbooks,  along  with  a  recommended  documentation  struc¬ 
ture.  Phase  III  deals  with  the  end  products  of  this  con¬ 
tract,  which  include  either  outlines  or  drafts  of  proposed 
changes  for  existing  documents  and  outlines  and  summaries 
for  proposed  new  documents. 
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PHASE  I 


Figure  1-1.  Technical  Approach 
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This  Final  Technical  Report  for  Part  I  of  the 
Testability/Diagnostics  Encyclopedia  Program  documents  the 
results  of  the  above  effort. 

Section  II  of  this  report  defines  some  important  terms 
used  in  the  report.  Section  III  describes  the  technical  ap¬ 
proach,  and  Section  IV  delineates  the  conclusions  and  recom¬ 
mendations,  including  the  Document  Improvement  Reports, 
emanating  from  this  analysis  effort.  Section  V  is  a  sum¬ 
mary  . 
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II.  DEFINITIONS. 


To  fully  understand  the  contents  of  this  report,  it  is 
necessary  to  understand  the  definitions  of  four  terms  used 
throughout  this  report.  The  first  term  is  "testability," 
which  is  defined  as  "a  design  characteristic  which  allows 
the  status  of  the  unit  to  be  confidently  determined  in  a 
timely  fashion."  Therefore,  testability  may  be  regarded  as 
inherent  to  the  unit's  design. 

"Diagnostics"  is  defined  as  "the  act  of  performing  and 
the  techniques  used  in  determining  and  isolating  the  cause 
of  malfunctions." 

"Integrated  Diagnostics"  is  defined  as  "a  structured 
process  which  maximizes  the  effectiveness  of  diagnostics  by 
integrating  pertinent  elements,  such  as  testability, 
automatic  and  manual  testing,  training,  maintenance  aiding, 
and  technical  information,  as  a  means  for  providing  a  cost- 
effective  capability  to  detect  and  unambiguously  isolate  all 
faults  known  or  expected  to  occur  in  weapon  systems  and 
equipment  in  order  to  satisfy  weapon  system  mission 
requi rements . " 

"Diagnostic  capability"  refers  to  all  the  capabilities 
associated  with  the  detection  and  isolation  of  faults,  in¬ 
cluding  automatic  and  manual  testing,  training,  maintenance 
aiding,  and  technical  information. 

It  is  important  to  understand  that  Integrated  Diagnos¬ 
tics  is  a  structured  design  process  for  acquiring  a  diagnos¬ 
tic  capability. 
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III. 


TECHNICAL  APPROACH. 


A.  Identification  of  Documents  to  be  Considered. 

The  Statement  of  Work  for  this  program  identified  a 
number  of  standards  and  handbooks  in  the  fields  of 
reliability,  maintainability,  system  engineering,  logistics, 
and  testing  which,  as  a  minimum,  were  to  be  examined.  As 
part  of  Phase  I  of  this  contract,  the  DoD  Index  of 
Specifications  and  Standards  was  thoroughly  reviewed  and  a 
number  of  additional  documents  were  added  to  the  scope  of 
the  project. 

In  addition  to  this,  14  Standardization  Areas  were  iden¬ 
tified.  A  number  of  Standardization  Plans,  either  existing 
or  under  preparation,  were  available.  Information  contained 
in  these  plans  would  assist  in  determining  future  standard¬ 
ization  activities  related  to  revising  existing  documents 
and  plans  for  developing  new  documents.  As  a  result,  the 
scope  of  a  number  of  documents  to  be  developed  in  the  near 
future  were  included  in  this  project.  Of  the  14  Standard¬ 
ization  Areas  identified,  only  Human  Factors,  Main¬ 
tainability,  and  Reliability  had  standardization  plans  which 
were  both  pertinent  and  up-to-date. 

Appendix  A  is  a  listing  of  existing  and  planned  docu¬ 
ments  which  were  incorporated  in  this  analysis.  Although 
the  contract's  Statement  of  Work  (SOW)  identified  a  minimum 
core  of  14  standards  and  two  handbooks,  the  number  of  docu¬ 
ments,  which  conceivably  could  have  had  testability  and 
diagnostic  implications,  was  much  higher.  As  summarized  in 
Figure  3-1,  54  standards  and  28  handbooks  and  guides  were 

included.  Twenty  percent  of  the  standards  reviewed  require 
major  modifications  to  adequately  deal  with  testability  and 
diagnostics.  Another  18%  require  minor  revisions.  Only  a 
small  percentage  of  handbooks  and  guides  require  revision 
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NUMBER  OF  DOCUMENTS 


MILITARY 

STANDARDS 


MILITARY 

HANDBOOKS 


REVIEWED 

54 

28 

MAJOR  REVISION  REQUIRED 

11 

3 

MINOR  REVISION  REQUIRED 

10 

0 

NO  REVISION  REQUIRED 

33 

25 

Figure  3-1.  Numerical  Summary  of  Analysis  Results 


B.  Establishing  Requirements  for  Standards  and  Handbooks. 

As  part  of  Phase  I  activity,  the  requirements  for  stan¬ 
dards  and  handbooks  were  established.  This  analysis  was  ap¬ 
proached  without  referring  to  existing  documents.  Rather, 
the  diagnostic  activities  relating  to  the  weapon  system  life 
cycle  were  analyzed  in  relation  to  standards  and  handbook 
requirements.  The  top-down  approach  used  for  determining 
these  documentation  requirements  is  described  in  Appendix  B 
of  this  report.  The  analysis  performed  during  Phase  I 
resulted  in  establishing  the  eight  basic  documentation  re¬ 
quirements  described  below. 

Requirement  #1  —  Establishing  Diagnostic  Requirements  and 

Allocating  These  Requirements  for 
System,  subsystem,  and  Unit  Levels. 

A  need  exists  for  establishing  diagnostic  requirements 
and  allocating  these  requirements  at  system,  subsystem,  and 
unit  levels.  This  includes: 

o  mranslating  weapon  system  mission  and  performance 
requirements  to  diagnostic  requirements 

o  Means  for  trading-off  various  diagnostic  elements  to 
achieve  optimization 

o  Allocating  diagnostic  requirements  to  these  diagnostic 
elements 

o  Means  for  specifying  diagnostic  requirements  in 
contractual  documents. 
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Requirement  #2  —  Describing  Various  Testability/ 

Diagnostic  Tasks  Which  Must  Be  Under¬ 
taken  During  Each  Phase  of  Weapon  System 
Acqu i s i t i on . 


A  requirement  exists  for  programmatic-type  documents 
which  describe  the  various  tasks  which  must  be  undertaken 
during  each  phase  of  weapon  system  acquisition.  This  re¬ 
quirement  includes  a  means  for  integrating  the  diagnostic 
requirements  that  are  of  concern  to  the  system  engineering, 
logistic  support,  reliability,  and  maintainability  of  weapon 
systems.  The  requirement  addresses  both  planning  and 
program  implementation. 


Requirement  #3  —  Designing  the  Diagnostic  Capability. 

A  requirement  exists  to  assure  proper  design  of  the 
weapon  system's  diagnostic  capability.  This  includes: 

o  Integrating  the  entire  diagnostic  design  process, 

beginning  with  the  establishment  of  diagnostic  require¬ 
ments  and  extending  through  the  maturation  of  the  diag¬ 
nostic  capability 

o  Assuring  that  design  criteria  exists  for  both  the  hard¬ 
ware  and  the  software  which  constitute  the  diagnostic 
capabi 1 i ty . 

Requirement  #4  --  Conducting  Design  Reviews. 

A  requirement  exists  to  conduct  a  number  of  technical 
reviews  and  audits  during  weapon  system  acquisition  to  as¬ 
sure  that  an  adequate  diagnostic  capability  is  attained. 


Requirement  #5  --  Analyzing  and  Assessing  the  Performance 

of  the  Diagnostic  Capability  (During 
Prime  System/Weapon  System  Design) . 

A  requirement  exists  to  analyze  and  assess  the  perfor¬ 
mance  of  the  diagnostic  capability  throughout  the  design  of 
the  weapon  system. 
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Requirement  #6  --  Assuring  the  Delivery  of  an  Adequate 

Diagnostic  Capability. 

A  requirement  exists  to  assure  the  delivery  of  an  ade¬ 
quate  diagnostic  capability  by  performing  and  conducting 
demonstration  and  maturation  programs  and  procedures.  The 
requirement  for  unambiguous  fault  detection  and  isolation  to 
the  lowest  replaceable  unit  at  each  maintenance  level  re¬ 
quires  a  comprehensive  re-look  at  these  evaluation  proce¬ 
dures  and  techniques. 

Requirement  #7  --  Collecting  and  Analyzing  Data  on  the 

Performance  of  the  Diagnostic  Capabil- 
ity. 

A  requirement  exists  for  establishing  procedures  for 
collecting  and  analyzing  data  on  performance  of  the  diagnos¬ 
tic  capability.  This  requirement  begins  in  the  early  stages 
of  weapon  system  development  and  continues  throughout  the 
life  of  the  weapon  system.  This  process  must  include  the 
requirement  for  the  compatibility  between  the  contractor's 
data  collection  system  and  the  military  service  collection 
system. 

Requirement  #8  —  Standardization  Definitions. 

A  requirement  exists  to  standardize  on  the  definitions 
utilized  in  acquiring  and  fielding  a  weapon  system  diagnos¬ 
tic  capability.  These  terms  must  be  standardized  in  order 
to  assure  unambiguous  understanding  of  diagnostic  require¬ 
ments  and  their  use  throughout  the  life  of  the  weapon  sys¬ 
tem  . 


Figure  3-2  depicts  the  relationship  of  the  first  seven 
requirements  to  the  weapon  system  life  cycle  (WSLC) . 

C.  Identifying/Scoping  of  Modifications  to  Existing  Docu¬ 
ments  and  Need  for  New  Documents. 

The  objectives  of  Phase  II  were  twofold.  The  first  was 
to  determine  the  effectiveness  of  the  individual,  existing 
documents  so  that  needed  revisions  to  these  documents  could 
be  identified.  The  second  was  to  determine  how  well  these 
documents  played  together  so  that  gaps  in  required  documen¬ 
tation  could  be 
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identified.  These  gaps  could  be  filled  by  revising  an  ex- 
existing  document  or  by  creating  a  new  one.  This  latter  ob¬ 
jective  deals  with  the  integration  of  documents  and  the 
traceability  among  documents.  Appendix  B  describes  this 
analysis  and  its  results. 

D.  Coordination  of  Phase  II  Effort  (Identification  and 
Scoping  of  Documents) . 

Before  embarking  on  Phase  III  of  this  program,  which 
addressed  detailed  modifications  and  outlines  of  documents, 
considerable  time  was  spent  in  coordinating  the  initial 
analysis  of  proposed  actions  with  government  and  industry 
personnel.  Copies  of  the  Phase  II  report  were  sent  to  the 
DoD  preparing  activities  for  each  document,  subsequent  to 
discussions  with  these  responsible  parties.  In  addition, 
coordination  with  industry  and  other  DoD  personnel  was 
achieved  through  two  meetings  with  the  National  Security  In¬ 
dustrial  Association's  (NSIA)  Integrated  Diagnostic  Group. 
The  results  of  these  two  meetings  are  contained  in  their 
report — a  copy  of  which  is  included  as  Appendix  C.  Their 
conclusions  and  recommendations  were  thoroughly  reviewed  and 
considered  during  the  Phase  III  effort. 
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IV.  FINDINGS,  CONCLUSIONS,  AND  RECOMMENDATIONS. 


The  output  of  Phase  III  activity  was  specific  on 
proposed  revisions  to  24  standards  and  handbooks.  Document 
Improvement  Reports,  which  are  located  at  the  end  of  this 
section  of  the  report,  were  prepared  for  each  of  these  24 
documents.  Fourteen  of  these  documents  require  major  revi¬ 
sions  and,  of  these  fourteen,  only  three  are  considered  to 
be  critical  (i.  e.,  MIL-STDs  2165,  415,  and  1521).  With  the 
exception  of  guidance  which  is  required  to  implement  the  in¬ 
tegrated  diagnostic  process,  no  new  documents  are  proposed. 

In  completing  the  final  phase  of  this  effort,  a  number 
of  key  findings  and  conclusions  were  evident,  which  threaded 
their  way  through  many  of  these  documents.  These  are  sum¬ 
marized  below. 


A.  Findings  and  Conclusions. 


o  Specification  of  the  Diagnostic  Capability. 


A  means  for  specifying  diagnostic  requirements 
in  RFPs ,  Statements  of  Work,  and  System  Specifica¬ 
tions  presents  a  formidable  problem.  The  National 
Security  Industrial  Association's  "Guidelines  for 
Preparation  of  Diagnostic  Requirements"  addresses 
the  problem  facing  both  the  government  and  the  con¬ 
tractor  program  manager  and  is  a  first  step  in  solv¬ 
ing  this  problem.  For  the  System  Specification, 
which  is  normally  written  by  the  contractor,  there 
are  almost  as  many  methods  for  specifying  diagnos¬ 
tics  as  there  are  contractors.  Virtually  none  of 
the  available  specification  methods  address  all  the 
diagnostic  elements.  Quantitatively  addressing 
technical  information  and  technical  orders,  quan¬ 
titatively  addressing  training  and  personnel  re¬ 
quirements,  and  quantitatively  addressing  perfor¬ 
mance  monitoring  requirements  are  examples  of  this 
problem.  Specification  must  be  made  at  prime  system 
level  and  as  the  design  progresses,  quantitative  al¬ 
locations  must  be  made  to  each  of  the  diagnostic 
elements.  In  addition,  provisions  for  prognostics, 
quality  assurance,  etc.,  must  be  addressed.  The  al¬ 
location  of  each  of  the  diagnostic  elements  in  rela¬ 
tion  to  100%  fault  detection  and  fault  isolation  re¬ 
quirements  at  each  maintenance  level  presents  an  ad¬ 
ditional  problem,  for  it  is  usually  a  combination  of 
diagnostic  elements  which  satisfy  a  specific  diag- 
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nostic  requirement.  For  example,  if  built-in  test, 
plus  a  technical  order,  are  utilized  to  fault  iso¬ 
late  to  a  given  unit,  what  percentage  is  allocated 
to  each?  Standard  methods  for  specifying  diagnostic 
requirements  need  to  be  developed  and  documented. 
Without  a  sound  methodology,  the  design  of  the  en¬ 
tire  diagnostic  capability  is  compromised. 

o  Integrating  the  "Ilities. " 

Providing  an  effective  diagnostic  capability  is 
based  on  sound  system  engineering  principles  and 
methods.  Thus  the  acquisition  of  a  suitable  diag¬ 
nostic  capability  is  based  on  the  system  engineering 
process  defined  in  MIL-STD-4 99A .  Emanating  from 
this  standard  are  the  logistic,  reliability,  main¬ 
tainability,  testability,  human  engineering,  and 
training  programmatic  standards,  which  define  tasks 
which  must  be  undertaken  in  relation  to  the  weapon 
system  acquisition  phases.  Although  each  of  these 
standards  relate  to  one  another  and  this  relation¬ 
ship  is  addressed  in  each  standard,  the  traceability 
throughout  the  system  engineering  process  for 
developing  the  diagnostic  capability  is  not  readily 
apparent.  Duplication  exists  among  these  standards. 
For  instance,  the  required  planning  documents  and 
the  demonstration  programs  inherently  lead  to 
duplication.  This  duplication  not  only  creates  un¬ 
necessary  paper  but,  if  not  controlled,  can  manifest 
itself  in  overlapping  engineering  activities.  A 
revolutionary  change  would  be  to  incorporate  the 
"ilities"  into  one  system  engineering  document. 
However,  at  present  there  are  some  promising  near- 
term  solutions.  One  solution  is  the  integration  of 
the  deliverables  required  by  the  consolidation  of 
Data  Item  Descriptions. 

o  Providing  a  Cohesive  Diagnostic  Design  Process. 

The  diagnostic  design  process  is  controlled  by 
a  number  of  military  standards,  handbooks,  and 
guides.  Among  these  are  standards  which  apply  to 
failure  modes  and  effects  analysis,  on-line  test, 
test  requirement  documents,  test  program  sets,  and 
technical  orders.  It  is  essential  that  the  diagnos¬ 
tic  design  process  integrate  and  utilize  the 
products  developed  under  reliability,  main¬ 
tainability,  logistic  support  analysis,  and  tes- 
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tability.  Generation  of  on-line  diagnostics  (e.  g., 
BIT,  test  point  placement,  in-flight  monitoring, 
maintenance-aiding)  is  particularly  fractionated. 
Methods  and  tools  for  testability  and  fault-tolerant 
design  are  required.  Vertical  testability  among  the 
various  maintenance  levels  is  oftentimes  neglected. 
Thus  the  main  objective  is  to  assure  that  a 
methodology  which  promotes  a  cohesive  diagnostic 
design  process  is  reflected  in  the  standards,  DIDs, 
handbooks,  and  guides. 

The  severity  of  this  problem  cannot  be  overem¬ 
phasized.  The  standards  which  control  the  diagnos¬ 
tic  design  process  are,  for  the  most  part,  adequate 
for  their  specific  purpose.  When  taken  collec¬ 
tively,  the  process  is  so  fractionated  that  effec¬ 
tive  implementation  is  virtually  impossible.  As  an 
example,  repeated  below  is  paragraph  4.7  of  MI  L- 
STD-756B. 

"4.7  Coordination  of  Effort.  Reliability  and 
other  organizational  elements  shall  make  coincident 
performance  and  use  of  the  reliability  models  and 
predictions.  Consideration  shall  be  given  to  the 
requirements  to  perform  and  use  the  reliability 
models  and  predictions  in  support  of  a  reliability 
program  in  accordance  with  MIL-STD-785,  main¬ 
tainability  program  in  accordance  with  MIL-STD-470, 
safety  program  in  accordance  with  MIL-STD-882,  sur¬ 
vivability  and  vulnerability  program  in  accordance 
with  MIL-STD-2072,  logistics  support  analysis  in  ac¬ 
cordance  with  MIL-STD-1 3 8 8 ,  maintenance  plan 
analysis  (MPA)  in  accordance  with  MI L-STD-2 0 8 0  , 
fault  diagrams  analysis  in  general  accordance  with 
MIL-STD-1591 ,  and  other  contractual  provisions." 

in  this  paragraph  alone,  eight  standards  are 
mentioned,  most  of  which  have  some  testability/' 
diagnostic  implications.  Add  to  this  list  a  number 
of  other  standards,  such  as  MIL-STDs  415,  1519, 
1326,  2076,  1843,  and  2077  and  the  problem  is  in¬ 
tensified.  Assuming  there  is  a  chronological  order 
in  the  application  of  these  standards  and  the 
products  they  produce,  if  a  link  in  the  chum 
breaks,  the  process  is  no  longer  applicable.  It  is 
no  wonder  that  often  the  produced  products  (e.  g., 
FMEAs ,  TRDs ,  and  LSARs)  become  merely  data 
deliveries  to  the  government  and  provide  little  in- 
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put  to  the  system  engineering 
producer  of  these  products  can 
quirement,  his  delivery  reflects 


process.  If  the 
see  no  urgent  re- 
this  perception. 


The  same  is  true  of  guidance  information. 
There  are  guidance  documents  which  are  written  for 
each  of  the  diagnostic  elements  (e.  g.,  tes¬ 
tability,  ATE),  but  at  present  none  that  completely 
promote  cohesiveness  of  the  diagnostic  design 
process . 

o  Effects  of  New  Weapon  System  Architectures. 

New  weapon  system  architecture,  which  incor¬ 
porates  dynamic  reconfigurability,  complex  redun¬ 
dancy,  and  graceful  degradation,  such  as  that  which 
is  depicted  in  the  Joint  Integrated  Avionics  Plan, 
requires  a  rethinking  of  the  standards  and  guidance 
used  in  the  diagnostic  design  process.  The  diagnos¬ 
tic  design  must  relate  to  the  mission  profile  and 
the  relevant  repair  and  operational  strategies. 


o  Effects  of  Advanced  Technologies. 

A  host  of  new  or  integrated  technologies  will 
have  a  major  effect  on  the  documentation  require¬ 
ments.  Artificial  intelligence  applications  to  such 
diagnostic  elements  as  expert  systems  and  smart  BIT 
will  require  a  re-look  at  some  well-established 
processes  and  procedures.  The  operational  scenario 
of  a  system  which  "learns"  and  whose  software  is 
modified  in  real-time  is  an  example.  Whether  or  not 
to  utilize  present  configuration  management  prin¬ 
ciples  as  this  software  changes,  and  utilize  this 
same  software  for  other  aircraft  "tail"  numbers, 
other  LRUs,  etc.,  must  be  addressed.  Standard¬ 
ization  of  languages  utilized  to  develop  such  diag¬ 
nostic  programs  and  methods  for  generating  technical 
information  must  be  considered.  Electronic  delivery 
of  technical  information  and  the  display  of  this  in¬ 
formation  for  maintenance  and  training  purposes  re¬ 
quires  standard  approaches.  VLSI  and  VHSIC  present 
more  complex  support  problems.  The  integration  of 
diagnostic  elements,  such  as  are  proposed  in  the  In¬ 
tegrated  Maintenance  Information  System  (IMIS)  re¬ 
quire  design  guidance. 
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Other  existing  technologies,  which  are  not  ade 
quately  addressed  by  present  standards,  are  perfor¬ 
mance  monitoring  and  in-flight  monitoring,  prognos¬ 
tics  and  preventive  maintenance,  man/machine  inter¬ 
faces,  and  sensor  technology. 

o  Test,  Evaluation,  and  Maturation. 

Effective  development  of  a  diagnostic 
capability  requires  that  testing  and  evaluation  of 
this  capability  proceed  concurrently  with  weapon 
system  development,  in  an  orderly  and  planned  time- 
phased  manner.  It  is  also  necessary  that  a  goal  be 
established  to  provide  evaluation  of  the  entire 
diagnostic  capability  at  the  time  the  prime  system 
demonstrations,  test,  and  evaluations  occur. 
Maturation  of  this  diagnostic  capability  must  be  an 
integral  part  of  the  diagnostic  demonstration 
process . 

The  foundation  for  diagnostic  demonstrations  is 
located  in  the  System  Specification.  Not  only  must 
the  means  for  demonstrating  the  diagnostic 
capability  be  established  at  that  time,  but  the 
definition  of  a  failure  must  also  be  established. 
When  demonstrating  graceful  degradation  through 
fault-tolerant  design,  reconfigurability,  redun¬ 
dancy,  and  performance  monitoring,  a  failure  may  be 
defined  as  causing  the  mission  and  performance  re¬ 
quirements  of  the  prime  system  to  be  compromised. 
This  definition  of  failure  then  becomes  an  important 
part  in  the  demonstration  process. 

Another  aspect  which  is  recommended  be  intro¬ 
duced  is  that  of  diagnostic  growth,  similar  in  con¬ 
cept  to  the  already  established  reliability  growth, 
except  with  an  entirely  different  means  for  specify¬ 
ing  and  demonstrating  this  growth.  This  diagnostic 
growth  is  an  integral  part  of  the  maturation 
process.  Some  combination  of  statistical  methods, 
test  analyze  and  fix,  and  contractual  incentives  can 
be  used  to  implement  this  concept.  GFE  and  CFE  re¬ 
quire  additional  attention  during  this  maturation 
period.  The  statement  of  requirements  for  these 
items  must  be  verified  and,  if  not  satisfying  those 
requirements,  alternative  solutions  must  be 
prov  ided . 
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Demonstration  programs  are  required  for  logis¬ 
tics,  reliability,  maintainability,  human  engineer¬ 
ing,  etc.  The  standards  urge  that  each  of  these 
demonstrations  take  advantage  of  the  others. 
However,  more  needs  to  be  done  to  assure  these 
various  demonstrations  take  advantage  of  one  another 
and  effect  combinations  when  feasible. 

o  Data  Collection  and  Analysis. 

The  major  programmatic  standards  require  that  a 
Data  Collection  and  Analysis  of  their  program  be  es¬ 
tablished  and  pursued.  This  reporting,  tracking, 
and  measurement  of  the  diagnostic  capability  perfor¬ 
mance  is  required  throughout  the  length  of  the  con¬ 
tract  and  is  an  integral  part  of  the  diagnostic 
maturation  process.  With  the  exception  of  MIL-STD- 
1388-2,  there  is  little  or  no  information  on  what 
and  how  data  is  collected.  Although  many  Air  Force 
data  systems  exist  or  are  in  the  planning  stages, 
none  satisfy  the  total  diagnostic  requirement.  Not 
only  is  this  data  required  to  measure  the  perfor¬ 
mance  of  the  diagnostic  capability,  but  it  provides 
firm  baseline  data  for  the  design  of  the  next  gener¬ 
ation  of  prime  systems.  Means  must  also  be  estab¬ 
lished  to  assure  compatibility  between  contractor¬ 
generated  data  bases  and  service  data  bases. 

B.  Recommendations. 

These  key  findings  and  conclusions  are  reflected  in  the 
24  Document  Improvement  Reports  located  at  the  end  of  this 
section.  It  is  recommended  that  these  improvements  be  in¬ 
itiated  by  the  appropriate  DoD  preparing  activities.  As  in¬ 
dicated  previously,  this  report  contains  no  recommendations 
for  preparing  new  documents,  with  the  exception  of  guidance 
for  implementing  the  integrated  diagnostic  process.  Rather, 
emphasis  was  placed  on  modification  and  consolidation  of  ex¬ 
isting  documents.  Critical  to  the  implementation  of  this 
document  improvements  are  the  following  three  recommenda¬ 
tions  : 
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Programmatic  Diagnostic  Standard 


There  is  no  question  that  a  programmatic 
standard  is  required,  which  addresses  the  entire 
testability/diagnostic  program.  MIL-STD-2165,  Tes¬ 
tability  Program  for  Electronic  Systems  and  Equip¬ 
ments,  presently  addresses  many  of  these  require¬ 
ments.  No  new  standard  is  proposed.  It  is  recom¬ 
mended  that  this  standard  be  expanded  to  cover  not 
only  the  design  characteristics  of  the  weapon  system 
itself  (e.  g.,  testability),  but  other  diagnostic 

elements  which  constitute  the  diagnostic  capability. 
The  standard  would  no  longer  be  limited  to  just 
electronics,  but  would  address  the  entire  weapon 
system.  The  intent  is  to  strengthen  the  design  and 
integration  portion  of  the  present  standard,  while 
lessening  the  requirements  for  program  monitoring 
and  control,  and  test  and  evaluation  by  utilizing 
other  existing  standards  to  perform  these  functions. 


o  Design  Criteria  Standard. 

As  cited  above,  there  are  a  number  of  standards 
which  control  the  diagnostic  design  process.  A  num¬ 
ber  of  these  standards  either  fully  or  partially  ad¬ 
dress  diagnostic  design  criteria.  This  design 
criteria  is  fractionated  and  often  obsolete.  MIL- 
STD-415D,  although  grossly  outdated,  currently  ad¬ 
dresses  design  criteria  for  test  provisions  for 
electronic  systems.  It  is  recommended  that  this 
standard  be  completely  rewritten  and  expanded  to  ad¬ 
dress  new  technologies,  new  system  architectures, 
and  the  weapon  system's  entire  diagnostic 
capability. 


o  Technical  Reviews  and  Audit  Standard. 


The  appendices  to  MIL-STD-1521B  describe  the 
requirements  for  the  conduct  of  technical  reviews 
and  audits  on  systems,  equipments,  and  computer 
software.  These  appendices  are  in  the  form  of 
checklists,  which  contain  the  purpose  of  the  par¬ 
ticular  technical  review/audit  and  a  list  of  items 
to  be  reviewed  at  each  one.  This  standard  is  only 
partially  complete,  in  relation  to  testability  and 
diagnostics.  Since  this  standard  is  the  only 
guidance  available  for  use  during  these  various 
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technical  reviews  and  audits,  it  is  important  that 
it  be  updated  to  provide  reviewers  with  appropriate 
guidance . 

In  addition,  the  following  two  recommendations  are  im¬ 
portant. 


o  Terminology  Standard, 


Standardization  of  terms  used  in  the  fields  of 
testability  and  diagnostics  is  important  in  the 
specification  and  implementation  of  testability  and 
diagnostics.  There  is  a  lack  of  standardization  and 
thus  in  the  understanding  of  these  terms. 
Presently,  there  are  two  standards  which  are  dedi¬ 
cated  to  defining  diagnostic  and  testability  terms. 
They  are  MIL-STDs  721  and  1309.  MIL-STD-721  best 
fills  the  need  for  standardization  terms,  but  it 
should  be  expanded  to  cover  an  additional  54  terms. 
It  is  also  recommended  that  MIL-STD-1309  be  retained 
as  a  dictionary  of  words  which  are  either  not 
preferred  or  are  less  pertinent. 


o  Obsolete  Standards. 


During  the  course  of  this  program  it  became  ap¬ 
parent  that  a  number  of  standards  were  either  in  the 
process  of  becoming  obsolete,  were  already  obsolete, 
or  would  become  obsolete  the  recommended  modifica¬ 
tion  to  present  standards  were  implemented.  It  is 
proposed  that  the  following  five  standards  either  be 
canceled  or  not  used. 


MIL-STD-1326  —  Test  Point,  Test  Point 

Selection  and  Interface  Re¬ 
quirements  for  Equipments 
Monitored  by  Shipboard  On- 
Line  Automatic  Test  Equipment 

MIL-STD-2076  —  Unit  Under  Test  Compatibility 

With  Automatic  Test  Equipment, 
General  Requirements  For 

MIL-STD-2084 (AS)  --  Maintainability  of  Avionic  and 

Electronic  Syustems  and  Equip¬ 
ment,  General  Requirements  For 
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MIL-STD-2080A  —  Maintenance  Plan  Analysis  for 

Aircraft  and  Ground  Support 
Equ i pment 

DOD-STD-1701  --  Hardware  Diagnostic  Test 

Systems  Requirements 

In  addition,  consolidation  of  MIL-STD-1519 
(Test  Requirements  Documents,  Preparation  Of)  and 
MIL-STD-1345  (Test  Requirements  Document,  Prepara¬ 
tion  Of)  is  proposed. 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  1 


REQUIREMENT  TITLE:  ESTABLISHING  DIAGNOSTIC  REQUIREMENTS  AND 

ALLOCATING  THESE  REQUIREMENTS  FOR 
SYSTEM,  SUBSYSTEM,  AND  UNIT  LEVELS 


DOCUMENT  NUMBER:  MIL-STD-756B 


TITLE: 


RELIABILITY  MODELING  AND  PREDICTION 


EXISTING:  X 


PLANNED: 


STANDARDIZATION 

AREA:  RELIABILITY 


PREPARING  NAVAIR  51122,  WASHINGTON,  D.  C. 

ACTIVITY:  JOHN  COOK,  NAEC ,  201-323-7458 

DOCUMENT  PURPOSE: 


This  standard  establishes  uniform  procedures  and  ground 
rules  for  the  preparation  of  mission  reliability  and  basic 
reliability  models  and  predictions  for  electronic,  electri¬ 
cal,  electromechanical,  mechanical,  and  ordnance  systems  and 
equipment.  A  mission  reliability  prediction  estimates  the 
probability  that  an  item  will  perform  its  required  functions 
during  the  mission  period. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

MI L-STD-7 56B  has  a  major  impact  on  testability  and 
diagnostics.  The  standard  defines  mission  reliability  equa¬ 
tions  which  are  key  to  testability  and  diagnostic  tradeoff 
decisions.  Introduction  of  diagnostics  via  a  system  en¬ 
gineering  approach  mandates  that  mission  measures  be  used  to 
provide  criteria  for  tradeoffs.  MIL-STD-756B  is  an  impor¬ 
tant  standard  in  that  it  defines  the  reliability  in  terms  of 
mission  reliability,  as  well  as  unit  reliability  at  the  sub¬ 
system  level.  Testability  parameters,  such  as  partitioning, 
observability,  and  controllability  will  influence  these 
reliability  decisions. 
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GENERAL  RECOMMENDATIONS: 


The  methodology  to  be  incorporated  into  MIL-STD-756B 
should  include  the  capability  to  analyze  all  modern  equip¬ 
ment  configurations.  This  should  address  the  following: 

o  Complex  redundancy,  reconf igurable  elements,  and 
configurations  which  allow  for  graceful  degradation 

o  The  capability  to  perform  the  aforementioned 

analyses  through  all  phases  of  a  mission  profile, 
utilizing  impacts  of  these  relevant  repair  and 
operational  strategies 

o  The  capability  for  analyses  to  be  sensitive  to  all 
testability/diagnostic  parameters,  including  those 
generated  by  BIT  and  embedded  support,  such  as  false 
alarms  and  on-line  fault  detection. 

SPECIFIC  MODIFICATIONS: 

Reliability  modeling  and  prediction  must  support 
tradeoffs  and  assessment  of  the  impact  of  the  dignostic  ele¬ 
ments  on  overall  weapon  system  reliability.  Current  mission 
reliability  models  deal  well  with  effectiveness  measures 
over  mission  time  demands  and  handle  complex  systems  which 
take  into  account  redundancy.  MIL-STD-756B  outlines  the 
mathematics  which  can  be  used  for  manually  generating  these 
models.  Its  information  is  generic  and  provides  a  basis  for 
current  models  and  for  addressing  reconfigurability. 

The  standard  is  not  sensitive  to  testability  and  diag¬ 
nostics  parameters  in  that  these  parameters  affect  the  ex¬ 
tent  to  which  a  system  is  initially  degraded  due  to  un¬ 
detected  or  unrepaired  failures  from  preceding  missions  and 
the  extent  to  which  a  failure  can  be  corrected  during  a  mis¬ 
sion. 


With  the  emergence  of  VHSIC  technology,  new  prime  sys¬ 
tem  architectures  focus  on  extremely  complex  designs  which 
allow  continued  operations,  with  full  capability  over  mis¬ 
sion  times  by  "reconfigurability"  of  the  line  replaceable 
modules  (LRM) .  In  other  words,  systems  are  being  designed 
to  be  "fault  tolerant"  in  that  they  allow  a  percentage  of 
failures  before  the  unit  is  considered  to  fail.  The  extent 
to  which  these  faults  can  be  detected,  diagnosed,  and  cor¬ 
rected  can  have  a  major  impact  on  the  success  of  the  next 
mission. 
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To  address  these  needs,  three  basic  changes  to  MIL- 
STD-756B  are  recommended:  incorporation  of  information  for 
repeated  mission  reliability  modeling,  information  about  the 
mathematics  for  transitions  from  mission  phase  to  mission 
phase,  including  an  equipment  turn-around  phase,  and  infor¬ 
mation  about  modeling  testability/diagnostics-related  repair 
or  replacement.  The  changes  are  outlined  below: 

TASK  SECTION  100  RECOMMENDATIONS 

insert  paragraph  2.2A: 

2.2A  Repeated  Mission  Reliability  Model.  A  paragraph 
defining  a  model  which  incorporates  testability  and 
diagnostic  elements  impacts  on  mission  success,  also 
taking  into  account  fault  detection  and  repair  during 
the  previous  mission  and  the  turn-around  time  between 
missions. 

Insert  paragraph  2.3.9: 

2.3.9  Reconf igurable  and  Complex  Mission  Requirements. 
A  paragraph  identifying  how  and  where  to  describe  mis¬ 
sion  reliability  requirements  which  are  not  represent¬ 
able  as  series  and  parallel  blocks  in  a  block  diagram. 

Insert  paragraph  2.5: 

2 . 5  Computing  the  Impact  of  Testability  and  Diagnostic 
Parameters.  A  paragraph  describing  what  testability 
factors  could  be  used  in  computing  Repeated  Mission 
Reliability  and  describing  how  to  incorporate  them  in  a 
Repeated  Mission  Reliability  Model. 
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TASK  SECTION  102  RECOMMENDATIONS 
Insert  paragraph  3.5: 

3.5  The  contractor  shall  develop  and  maintain  a 
Repeated  Mission  Reliability  Model  for  each  configured 
item  required  to  perform  the  mission  functions.  One  or 
more  paragraphs  shall  elaborate  on  the  traceability  of 
information,  consistency  of  nomenclature,  capability, 
and  other  aspects  of  the  Repeated  Mission  Reliability 
Model . 

METHOD  1001  RECOMMENDATIONS 
Insert  paragraph  2.3: 

2.3  Fault  Detection,  Diagnostics,  and  Repair.  One  or 
more  paragraphs  that  provide  the  underlying  equations 
to  be  used  in  accounting  for  testability  and  diagnostic 
impacts,  both  during  a  mission  and  during  a  turn-around 
period  between  missions  in  order  to  be  able  to  compute 
Repeated  Mission  Reliability. 

METHOD  1002  RECOMMENDATIONS 

Insert  paragraph  2.3: 

2.3  Fault  Detection,  Diagnostics,  and  Repair.  One  or 
more  paragraphs  that  provide  the  underlying  equations 
to  be  used  in  accounting  for  testability  and  diagnostic 
impacts,  both  during  a  mission  and  during  a  turn-around 
period  between  missions  in  order  to  be  able  to  compute 
Repeated  Mission  Reliability. 

METHOD  1003  RECOMMENDATIONS 

Insert  paragraph  2.3: 

2.3  Fault  Detection,  Diagnostics,  and  Repair.  One  or 
more  paragraphs  that  provide  the  underlying  equations 
to  be  used  in  accounting  for  testability  and  diagnostic 
impacts,  both  during  a  mission  and  during  a  turn-around 
period  between  missions,  in  order  to  be  able  to  compute 
Repeated  Mission  Reliability. 
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METHOD  1004  RECOMMENDATIONS 


Insert  paragraph  2.3: 

2.3  Fault  Detection/  Diagnostics,  and  Repair.  One  or 
more  paragraphs  that  provide  the  underlying  equations 
to  be  used  in  accounting  for  testability  and  diagnostic 
impacts,  both  during  a  mission  and  during  a  turn-around 
period  between  missions,  in  order  to  be  able  to  compute 
Repeated  Mission  Reliability. 

METHOD  1005  RECOMMENDATIONS 

Insert  Methods  Section  1005: 

METHOD  1005  MISSION  PHASE  TRANSITIONS  AND  SYSTEM 
HIERARCHY.  A  section  that  provides  the  procedures  for 
computing  the  initial  conditions  of  each  new  phase  from 
the  preceding  phase  and  for  using  the  different  com¬ 
putation  methods  in  modeling  a  systems  hierarchy  from 
component,  to  module,  to  box,  to  subsystem,  to  system. 

REPEATED  MISSION  RELIABILITY  PREDICTION 

Insert  Section  203,  Repeated  Mission  Reliability  Pre¬ 
diction: 

TASK  203  REPEATED  MISSION  RELIABILITY  PREDICTION. 
Several  paragraphs  identifying  its  purpose  and  rational 
and  giving  requirements  to  be  satisfied  in  its  develop¬ 
ment.  It  should  emphasize  the  model's  use  in  relating 
testability  and  diagnostic  requirements  to  mission 
needs  and  in  validating  testability  and  diagnostic  al¬ 
locations  . 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  1 

REQUIREMENT  TITLE:  ESTABLISHING  DIAGNOSTIC  REQUIREMENTS  AND 

ALLOCATING  THESE  REQUIREMENTS  FOR 
SYSTEM,  SUBSYSTEM,  AND  UNIT  LEVELS 

DOCUMENT  NUMBER:  MIL-STD-1591 

TITLE:  ANALYSIS/SYNTHESIS  OF  ON-AIRCRAFT,  FAULT 

DIAGNOSIS,  SUBSYSTEMS 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  MISCELLANEOUS 

PREPARING  RADC/RBE-2 ,  ROME,  NY 

ACTIVITY:  DAVE  GARAFALO,  315-330-3476 

DOCUMENT  PURPOSE: 

The  purpose  of  MIL-STD-1591  is  to  provide  uniform 
criteria  for  conducting  trade  studies  for  determining  op¬ 
timum  design  for  on-aircraft  fault  detection/isolation  sys¬ 
tems  . 

TESTABILITY/DIAGNOSTIC  IMPACT: 

MIL-STD-1591  is  a  key  document  because  it  is  the  only 
document  which  attempts  to  define  this  type  of  trade-off 
procedure.  At  this  time,  no  adequate  trade-off  procedure  or 
allocation  techniques  exist  which  are  based  upon  mission 
needs . 

GENERAL  RECOMMENDATIONS: 

This  should  be  a  companion  document  to  MIL-STD-756. 
When  modified,  MIL-STD-756  will  translate  weapon  system  mis¬ 
sion  requirements  into  diagnostic  requirements  and  will 
serve  to  allocate  these  requirements  down  to  each  diagnostic 
element.  There  is  a  need  to  relate  these  allocations  to 
cost  and  manpower.  At  present,  MIL-STD-1591  is  aimed  at  op¬ 
timizing  the  design  for  on-aircraft  fault 
detect i on/ i sol  at  ion.  The  recently  completed  RADC  task. 
Tools  for  Integrated  Diagnostics,  lays  the  groundwork  for  an 


initial  "most"  cost-effective  diagnostic  mix  and  provides  a 
general  method  for  assessing  the  cost  and  effectiveness  of 
the  diagnostic  mix  as  the  design  progresses  throughout  the 
Full-Scale  Development  Phase.  The  diagnostic  mix,  in  this 
case,  is  composed  of  built-in,  external,  and  manual 
resources  for  an  electronic  system. 

There  is  no  question  that  more  research  and  development 
is  required  to  deal  with  this  trade-off  function  prior  to 
revision  of  this  MIL-STD.  Major  emphasis  in  this  RDT&E  ef¬ 
fort  should  consider  the  following: 

o  Trade-off  procedures  which  address  all  diagnostic 
elements  (i.  e.,  testing,  training,  and  technical 
information) 

o  Trade-off  methodologies  which  can  be  applied 
throughout  weapon  system  development 

o  Trade-off  methodologies  which  address  all  types 
of  weapon  systems  and  both  electronic  and  non¬ 
electronic  applications. 

In  addition,  the  NSIA  Integrated  Diagnostics  Group 
recommended  that  this  type  document  be  issued  as  a  handbook, 
since  the  trade-off  methodologies  need  not  be  mandatory,  but 
rather  for  guidance. 

SPECIFIC  MODIFICATIONS: 

As  cited  above,  more  research  and  development  effort  is 
required  in  relation  to  trade-off  procedures  and  methods. 
The  military  services  are  presently  supporting  work  in  this 
area,  and  they  plan  to  sponsor  additional  work  in  the  near 
future.  It  is  suggested  that  modifications  to  MIL-STD-1591 
be  postponed  until  sufficient  work  has  been  completed  to 
permit  substantial  improvements  in  this  standard.  This  will 
provide  RADC  with  the  necessary  basis  for  an  integrated 
trade-off  procedures  document. 
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An  outline  of  recommended  changes  to  MIL-STD-1591  fol¬ 
lows.  These  changes  expand  the  scope  of  the  document  from 
aircraft  to  generic  systems,  fit  the  application  of  this 
document  into  the  framework  of  MIL-STD-2165 ,  satisfy  mission 
needs  and  constraints  through  application  of  the  revised 
MIL-STD-756B,  and  expand  the  scope  of  the  model  to  take  into 
account  all  elements  of  each  alternative  diagnostic 
capability.  These  changes  are  outline  below: 

Modify  paragraph  1.1,  Purpose 

The  purpose  of  the  document  should  be  expanded  to 
address  the  entire  diagnostic  capability,  not  just  on- 
aircraft  diagnostics. 

Modify  paragraph  1.2,  Application 

This  paragraph  should  be  modified  to  address  the 
entire  diagnostic  capability  at  all  maintenance  levels. 

Modify  paragraph  2.1, 

This  paragraph  should  reference  MIL-STD-2165  and 
MIL-STD-756B. 

Modify  Section  3,  Definitions  and  Abbreviations 

The  aircraft-specific  terms  and  definitions  should 
be  replaced  by  generic  words  and  definitions.  The  list 
of  terms  should  be  expanded  to  encompass  all  fault 
detection  and  diagnostic  components  used  for  weapon 
system  operation. 

Modify  paragraph  4.1,  Required  Parameters 

This  paragraph  should  identify  additional  areas  of 
consideration  related  to  repeated  mission  reliability, 
mission  resource  constraints  (volume,  power,  manpower, 
etc.),  and  impact  on  each  level  of  maintenance. 

Modify  paragraph  4.2,  Sequence  of  Work 

This  paragraph  should  be  modified  to  properly  in¬ 
troduce  computations  of  repeated  mission  reliability, 
needed  resources,  and  impacts  on  each  level  of  main¬ 
tenance.  This  modification  should  specify  that  alter¬ 
natives  which  are  unacceptable  for  meeting  any  of  the 
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mission  requirements,  or  which  exceed  resources  that 
can  be  made  available  for  operation,  must  be  removed 
from  consideration  before  the  cost  analysis  is  done. 

Modify  paragraph  5.1,  Determination  of  Conceptual  Op- 
tions 


This  paragraph  should  describe  additional  areas  of 
consideration  related  to  repeated  mission  reliability, 
mission  resource  constraints  (volume,  power,  manpower, 
etc.),  and  impacts  on  each  level  of  maintenance.  It 
must  be  expanded  to  encompass  all  fault  detection  and 
diagnostics- related  considerations. 

Modify  paragraph  5.2,  Selection  of  Best  Conceptual  Op- 
t  i  on 


This  paragraph  needs  to  identify  the  other 
parameters  besides  cost  which  affect  the  selection 
decision  and  describe  the  computations  or  reference 
documents  describing  the  computations  (e.  g.,  MIL-STD- 
756B)  . 


The  paragraph  5.2  cost  model  must  be  generalized 
to  encompass  all  fault  detection  and  diagnostic  ele¬ 
ments  used  for  system  operation  and  extended  to  address 
costs  other  than  the  maintenance  manpower  and  support 
equipment,  which  are  currently  considered.  Training, 
technical  information,  spares,  and  other  maintenance- 
level  workloads  need  to  be  considered. 

Modify  paragraph  5.3,  Procedure  for  Synthesizing  Detail 
Design  of  the  Selected  OBBIT  Concept 

This  paragraph  needs  to  be  modified  to  expand  OB¬ 
BIT  design  drivers  from  relative  fault  frequency  and 
detection  costs  to  include  mission  criticality  and  sur¬ 
vivability  factors. 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  1 

REQUIREMENT  TITLE:  ESTABLISHING  DIAGNOSTIC  REQUIREMENTS 

AND  ALLOCATING  THESE  REQUIREMENTS  FOR 
SYSTEM,  SUBSYSTEM,  AND  UNIT  LEVELS 

DOCUMENT  NUMBER:  MIL-STD-490A 

TITLE:  SPECIFICATION  PRACTICES 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  CMAN 

PREPARING  HQ  AFSC/PLEQ,  ANDREWS  AFB ,  WASHINGTON, 

ACTIVITY:  D.  C. 

301-981-2751 

DOCUMENT  PURPOSE: 

The  purpose  of  this  standard  is  to  establish  uniform 
practices  for  specification  preparation,  to  ensure  the  in¬ 
clusion  of  essential  requirements,  and  to  aid  in  the  use  and 
analysis  of  specification  content. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  specifications  stated  in  the  various  appendices  and 
DIDs  establish  the  requirement  for  specifying  the  configura¬ 
tion  item  functional  baseline,  allocated  baseline,  and 
product  baseline.  Diagnostic  elements  are  part  of  the 
baselines  and  must  be  defined  in  the  applicable  specifica¬ 
tions  . 

GENERAL  RECOMMENDATIONS: 

For  the  Prime  Item  Development,  the  Critical  Item 
Development,  and  System/Segment  Specifications,  it  is  recom¬ 
mended  that  a  paragraph  labeled  "Diagnostics"  be  included  in 
each  Section  3. 


SPECIFIC  MODIFICATIONS: 


Page  56,  Appendix  II,  Type  Bl,  Prime  Item  Development 
Specification,  after  Section  20. 3. 2. 2. 4C,  add  the  following: 

"20.3.2.4.1  Paragraph  3. 2. 4.1  Diagnostics.  The 
paragraph  shall  specify  the  quantitative  diagnostic 
capability  requirements.  The  requirements  shall  apply  to 
diagnostic  elements  at  all  planned  maintenance  levels.  Ex¬ 
amples  of  the  diagnostic  elements  are: 

o  Configuration  item  design  requirements  for  BIT/ 
Testability 

o  Conf igurati on  item  FD/FI  level  quantification 

o  Off-equipment  diagnostic  parametric  quantification 
in  terms  of  automatic  test  equipment/test  program 
sets  and  manual  test  equipment  performance 
requirements." 

Page  64,  Appendix  III,  Type  B2,  Critical  Item  Develop¬ 
ment  Specification,  after  paragraph  30.3.2.4,  add  the  fol¬ 
lowing  : 

"30.3.2.4.1  Paragraph  3. 2. 4.1  Diagnos t ics  .  This 
paragraph  shall  specify  the  quantitative  diagnostic 
capability  requirements.  The  requirements  shall  apply  to 
diagnostic  capability  in  the  planned  operational,  main¬ 
tenance,  and  support  environments  and  shall  be  stated  in 
diagnostic  capability  quantitative  parametric  terms." 

Page  48,  DI -CMAN-80008 ,  System/Segment  Specification, 
paragraph  6.2,  make  the  following  changes: 

Page  20,  insert  new  Paragraph  10.2.5.4.2.2, 

"Diagnostics"  and  renumber  each  subsequent  paragraph,  begin¬ 
ning  with  existing  paragraph  10.2.5.4.2.2  in  sequence . 
Each  paragraph  number  stated  in  the  first  sentence  of  each 
of  the  above-mentioned  paragraphs  must  also  be  increased  by 
one  integer  in  last  column. 

Add  new  paragraph  as  follows:  10.2.5.4.2.2 

"Diagnostics."  This  subparagraph  shall  be  numbered  3. 4. 2. 2 
and  shall  specify  quantitative  system  diagnostic  require¬ 
ments.  The  requirements  shall  apply  to  diagnostics  in  the 
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operational  mode  during  mission  performance,  to  0-Level 
maintenance  onboard  the  system  and  off-equipment  require¬ 
ments  at  the  O-I-D  levels  of  maintenance.  Examples  are: 

(a)  Status  monitoring  requirements  during  system 
operation 

(b)  Fault  detection/fault  isolation  levels  onboard 

(c)  Fault  detection/fault  isolation  levels  equipment 
at  each  maintenance  level." 


4-23 


DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 

REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE 
UNDERTAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-499A 

TITLE:  ENGINEERING  MANAGEMENT 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  MISCELLANEOUS 

PREPARING  HQ  AFSC/PLEQ,  ANDREWS  AFB ,  WASHINGTON, 

ACTIVITY:  D.  C. 

CAPT.  SYLVESTER,  301-981-2751 
HQ  AFSC/SDX ,  ANDREWS  AFB,  WASHINGTON, 

D.  C. 

MAJ.  KEN  MILLER,  301-981-3316 

DOCUMENT  PURPOSE: 

This  standard  provides  the  program  manager  with: 

(a)  Criteria  for  evaluating  engineering  planning  and 
output 

(b)  A  means  for  establishing  an  engineering  effort 
and  a  System  Engineering  Management  Plan  (SEMP) 

(c)  Task  statements  that  may  be  selectively  applies  to 
an  acquisition  program. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  effective  implementation  of  the  Integrated  Diagnos¬ 
tic  process  is  accomplished  through  the  inclusion  of  diag¬ 
nostic  requirements  in  the  SEMP.  Since  MIL-STD-499A 
provides  a  means  for  establishing  an  engineering  effort  and 
a  SEMP,  the  success  of  diagnostic  implementation  is  depend¬ 
ent  on  the  proper  implementation  of  the  requirements  stated 
in  MIL-STD-499A. 
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GENERAL  RECOMMENDATIONS: 


MIL-STD-499A  is  a  generic  standard  written  so  that  it 
defines  a  systems  engineering  management  process  that  is  ap¬ 
plicable  to  weapon  systems,  subsystems  (electronics,  en¬ 
gines,  support  equipment)  and  equipment.  Testability  and 
diagnostic  requirements,  while  not  specifically  addressed  in 
the  standard,  are  included  through  interpretation  as  part  of 
the  system  engineering  and  engineering  specialty  process. 
In  order  to  provide  needed  visibility  within  the  standard, 
it  is  recommended  that  the  word  "diagnostics"  be  inserted 
into  the  MIL-STD-499A  text,  as  stated  below:  These  inser¬ 
tions  do  not  impact  the  intent  or  the  requirements  of  the 
standard . 

SPECIFIC  MODIFICATIONS: 


Page  3,  paragraph  3.4:  In  the  first  sentence,  between  the 
words  "maintainability,"  and  "logistics",  insert 
"diagnostics , "  . 


Page 

4, 

paragraph  (d) : 

After  the  word 

"Ma i nta i nab  il i 

ty," 

inser 

t  " 

Diagnostics , " . 

Page 

5, 

paragraph  (n) :  Between  the  words 

"transpor tabil 

ity" 

and  " 

rel 

iability",  insert 

"diagnostics , " . 

Page 

8, 

paragraph  6.3: 

Seventh  (7th)  sentence,  after 

the 

word 

"ownership",  insert 

"  ,  diagnostics  ,  "  . 

4-25 


DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 

REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 


DOCUMENT  NUMBER:  MIL-STD-1388-1A 


TITLE: 


LOGISTICS  SUPPORT  ANALYSIS 


EXISTING:  X 

STANDARDIZATION 
AREA : 


PLANNED: 

MISCELLANEOUS 


PREPARING  ARMY,  AMXMD-EL,  LEXINGTON,  KY 

ACTIVITY:  JIM  CRABTREE,  606-293-3962 

ROBERT  SHIELD,  606-293-3962 

DOCUMENT  PURPOSE: 


The  purpose  of  MIL-STD-1388-1A,  the  Logistic  Support 
Analysis  process,  is  to  define  the  programmatic  requirements 
for  performing  support  and  supportability  tradeoffs  in  the 
weapon  system  development  process.  The  LSA  is  a  systematic 
and  comprehensive  analysis,  conducted  on  an  iterative  basis 
through  all  phases  of  the  system/equipment  life  cycle  to 
satisfy  supportability  objectives. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

Proper  implementation  of  the  LSA  process  should  include 
an  emphasis  on  testability  and  diagnostic  requirements,  be¬ 
cause  a  comprehensive  diagnostic  capability  may  decrease 
shortfalls  and  unnecessary  burdens  on  the  logistics  support 
structure.  It  is  necessary  to  consider  testability  and 
diagnostic  considerations  in  the  LSA  process. 
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GENERAL  RECOMMENDATIONS: 


MIL-STD-1388-1A  does  address  the  subject  of  testability 
and  diagnostics,  but  only  in  a  cursory  manner.  MIL-STD-1388 
contains  the  framework  for  the  necessary  analyses  to  be  con¬ 
ducted,  but  does  not  house  the  specific  details  or  level  of 
emphasis  required  for  a  comprehensive  analysis  relating  to 
testability  and  diagnostics.  It  is  recommended,  therefore, 
that  the  emphasis  be  placed  in  a  document  solely  dedicated 
to  diagnostics  (i.  e.,  MIL-STD-2165  Revised)  and  that  MIL- 
STD-1388  undergo  minor  revisions  to  increase  the  inclusion 
of  diagnostics  considerations  in  the  logistics  tradeoffs 
conducted.  A  strong  relationship,  with  specific  interfaces, 
between  the  diagnostics  standard  and  the  LSA  standard  should 
be  established  and  defined. 

SPECIFIC  MODIFICATIONS: 

Section  2.1:  Include  "MIL-STD-2165,  Testability  Program  for 
Electronic  Systems  and  Equipments,"  in  the  referenced  docu¬ 
ment  list. 

Task  303,  303.2.8:  Insert  "maintenance  aiding,"  after 

"automatic  testing,". 

Task  401,  401.2.3:  After  the  words  "support  and  test 

equipment,",  add  the  words  "maintenance  aids,". 

Task  402,  402.2.1:  After  the  words  "automatic  test 

equipment,",  add  the  words  "and  test  program  sets". 

Appendix  A,  paragraph  50.3.4.2c:  Replace  the  words 

"Maintainability  Program"  with  "Diagnostics/Testability 
Program  (MIL-STD-2165)". 

Appendix  A,  paragraph  50. 4. 2d:  After  the  words  "test 

program  sets  (TPS)",  add  the  words  "and  maintenance  aids". 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 


REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-785B 


TITLE: 


RELIABILITY  PROGRAM  FOR  SYSTEMS  AND 
EQUIPMENT  DEVELOPMENT  AND  PRODUCTION 


EXISTING:  X 


PLANNED: 


STANDARDIZATION 

AREA: 

PREPARING 

ACTIVITY: 


DOCUMENT  PURPOSE: 


RELIABILITY 

ASD/ENES,  WRIGHT-PATTERSON  AFB,  DAYTON, 
OH 

MARY  COURY,  513-255-6295 


This  standard  provides  general  requirements  and 
specific  tasks  for  reliability  programs  during  the  develop¬ 
ment,  production,  and  initial  deployment  of  systems  and 
equipment. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

A  weapon  system's  reliability  program  forms  the  basis 
for  maintainability,  testability,  and  diagnostic  programs. 
Faul t- tolerant  design  and  dynamic  reconfigurability  are  the 
basis  for  deferred  maintenance  and  thus  play  a  large  part  in 
determining  testability  and  diagnostic  requirements.  The 
failure  modes  and  effects  analyses,  which  are  a  substantial 
portion  of  a  reliability  program,  provide  a  significant  in¬ 
put  to  diagnostic  and  test  functions  by  identifying  likely 
ways  in  which  a  system  can  fail  and  the  criticality  of  a 
given  failure.  Throughout  reliability  design,  testing,  and 
demonstration,  this  diagnostic  capability  shall  not  only  be 
included  as  part  of  the  prime  system,  but  should  be 
evaluated  in  terms  of  its  troubleshooting  capability  (i.  e., 
ability  to  fault  detect  and  fault  isolate) . 
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GENERAL  RECOMMENDATIONS: 


At  present,  MIL-STD-785B  does  not  implement  MIL-STD- 
756,  Reliability  Modeling  and  Prediction.  Included  in  MIL- 
STD-756  are  mission  reliability  modeling  techniques  which 
can  be  applied  during  Concept  Exploration  and  Demonstration 
and  Validation  Phases  as  a  means  for  translating  weapon  sys¬ 
tem  mission  and  performance  requirements  into  reliability, 
maintainability,  testability,  and  diagnostic  requirements. 
In  addition,  the  standard  does  specifically  address  the 
reliability  of  the  diagnostic  capability. 

SPECIFIC  MODIFICATIONS: 

Add  paragraph  1.4,  Embedded  Diagnostics.  The  term  "system 
and  equipment"  includes  the  reliability  of  the  embedded 
diagnostic  capability  (e.  g.,  built-in  test,  maintenance 

aiding  and  training).  Therefore,  the  reliability  program 
described  in  this  standard  includes  the  embedded  diagnostic 
capability  as  an  integral  part  of  the  prime  systems  and 
equipment . 

Paragraph  2.1:  Add  "MIL-STD-756,  Reliability  Modeling  and 

Prediction"  to  Standards,  Military  Documents. 

Task  201,  paragraph  201.2.3:  Replace  the  words  "Appendix  A 
of  MIL-HDBK-217 "  with  "MIL-STD-756". 

Task  204  and  paragraph  50.2.3.1.2,  Appendix  A:  These  tasks 
should  recognize  that  FMECA  can  be  used  as  a  BIT  design 
tool.  Revise  paragraph  204.2.2  to  insert  the  words 
"designing  and"  in  front  of  the  words  "evaluating  the  effec¬ 
tiveness  of  built-in  test,". 

Paragraph  50.2.3.1.2:  Replace  the  last  sentence  of  this 

paragraph  to  read  as  follows: 

"FMECA  is  an  effective  tool  for  designing  the  diagnos¬ 
tic  capability  for  it  is  initial  step  in  generating  diagnos¬ 
tic  and  test  programs.  FMECA  is  also  an  effective  tool  for 
evaluating  the  effectiveness  of  built-in  uest." 

Paragraph  50,1.3.1.3:  After  the  word  "maintainability,"  in¬ 
sert  "diagnostics,  testability,”. 


► 


r 


Paragraph  50.2.4.1.1:  This  paragraph  should  be  written  to 
alert  the  reader  that  diagnostics  plays  a  key  role  in 
faul t- tolerant  design  for  it  provides  for  graceful  degrada¬ 
tion,  which  lessens  the  need  for  immediate  corrective  main¬ 
tenance.  However,  this  does  not  lessen  the  need  for  ac¬ 
curate  and  timely  fault  detection  and  isolation.  This  can 
be  accomplished  by  adding  the  words:  "including  the  design 
of  the  diagnostic  capability  which  is  required  to  recon¬ 
figure  the  system  and  to  isolate  the  failed  unit  when  cor¬ 
rective  maintenance  is  accompl i shed" ,  after  the  words, 
"Criteria  such  as  these  influence  the  design  and  operation 
of  almost  all  subsystems,". 


f 


i 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 


REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-470A 


TITLE: 


MAINTAINABILITY  PROGRAM  REQUIREMENTS 
(FOR  SYSTEMS  AND  EQUIPMENT) 


EXISTING:  X 


PLANNED: 


STANDARDIZATION 

AREA:  MAINTAINABILITY 


PREPARING 

ACTIVITY: 

DOCUMENT  PURPOSE: 


RADC/RBE-2,  ROME,  NY 
JERRY  KLION,  315-330-4726 


This  standard  provides  task  descriptions  for  main¬ 
tainability  programs. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

This  standard  is  very  closely  allied  to  MIL-STD-2165 , 
Testability  Program  for  Electronic  Systems  and  Equipments. 
Inasmuch  as  this  report  recommends  the  strengthening  of  the 
Task  200  series  in  MIL-STD-2165,  the  proposed  modifications 
to  MIL-STD-470A  are  minor. 


GENERAL  RECOMMENDATIONS: 

Cross-referencing  with  MIL-STD-2165  is  required  to  as¬ 
sure  compatibility. 

SPECIFIC  MODIFICATIONS: 

Paragraph  2.1:  Add  "MIL-STD-2165,  Testability  Program  for 
Electronic  Systems  and  Equipments"  to  standards,  military 
documents . 
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Paragraph  4.1.1:  Before  the  last  sentence,  add  "Testability 
Program  Requirements  for  Electronic  Systems  and  Equipments 
(MIL-STD-2165) ,  in  particular,  have  a  significant  interface 
with  a  majority  of  tasks  identified  in  this  standard." 

Task  101,  paragraph  101. 2. If:  After  each  use  of  the  word 

"maintainability,",  add  the  words  "testability  and 
diagnostics" . 

Task  103,  paragraph  103.2.2:  Add  the  following  to  the 

reviews  — 

"At  System  Requirements  Review: 

Results  of  trade  studies  leading  to  Preliminary  System 
Design  Concept." 

"At  System  Design  Review: 

(1)  Diagnostic  Content  of  Development  Specification 

(2)  Diagnostic  Maturation  and  Data  Collection  Plan 

(3)  System  Optimization  Tradeoffs 

(4)  Risk  Analysis 

(5)  Diagnostic  allocation." 

"At  Production  Readiness  Review: 

Results  of  Evaluation  of  Entire  Diagnostic  Capability." 

Task  103,  paragraph  103.2.2b,  at  the  Critical  Design  Review 
(CDR) ,  as  a  last  item  in  that  paragraph,  add,  "(10)  Inherent 
Testability  Assessment." 

Task  104,  General:  Add  the  words  "and  diagnostics"  after 

each  time  "maintainability"  is  used. 

Task  104,  paragraph  104.3.1b:  Replace  the  word  "test"  with 
"diagnostics" . 

Task  202,  paragraph  202.2.1:  Add  the  following  sentence  to 
the  end  of  the  paragraph:  "Inputs  from  Task  201,  MIL-STD- 

2165,  form  the  basis  for  diagnostic  allocation." 

Task  205,  paragraph  205.2.1:  Add  the  following  sentence  to 
the  end  of  the  paragraph:  "Inputs  from  Task  202,  MIL-STD- 
2165,  will  form  the  basis  for  testability  and  diagnostic 
analysis. " 
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Task  205,  paragraph  205.2.2.6:  To  further  elaborate  on  the 
items  to  be  considered  during  Maintainability  Analysis,  the 
paragraph  should  read  as  follows:  "Mixes  of  automatic,  semi¬ 
automatic,  built-in  and  manual  test,  maintenance  aids, 
manual  diagnostic  procedures  at  all  levels  of  repair  and 
their  associated  software  and  technical  costs,  skill  levels 
required  and  manpower  requirements,  and  acquisition  costs." 


Task  206,  paragraph  206.2.2.3(h):  The  list  of  guidelines 

and  policies  supplied  in  Maintainability  Design  need  to  be 
clarified  and  expanded.  Delete  and  replace,  with: 

" (h)  Testability  and  f ault-tolerant  design 
techniques . " 

Task  206,  paragraph  206.2.2.3:  Add  (n) ,  which  reads: 

"(n)  Interface  with  computer-aided  engineering  and 
computer-aided  design  techniques." 

Task  301,  paragraph  301.2.2(g):  After  the  words  "secondary 
failures,"  insert  "effectiveness  of  the  diagnostic 
capability" . 

APPENDIX  A: 

Table  A-l,  Application  Matrix.  If  Task  202,  Maintainability 
Allocations,  is  to  include  testability  and  diagnostic  al¬ 
locations,  then  the  general  usage  (g)  ,  cited  in  Table  A-l, 
should  also  be  applied  during  the  Demonstration  and  Valida¬ 
tion  Phase. 

Paragraph  40 . 1 . 5 . 1 .  If  .  :  After  the  word  "maintainability," 

add  the  words  "and  diagnostics". 

Paragraph  40.2.5.2:  After  each  time  the  word  "test"  is 
used,  add  the  words  "and  diagnostic".  Also  add  subparagraph 
as  follows: 

"(h)  Maintenance  aiding  and  other  diagnostic 
procedures . " 

Paragraph  4 0 . 2 . 5 . 2 . 6 ( new) :  insert  this  paragraph  after 

40.2.5.2.5  and  renumber  subsequent  paragraphs  accordingly: 
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"40.2.5.2.6  The  use  of  maintenance  aids,  including  diag¬ 
nostic  routines,  repair  procedures,  and  maintenance  histori¬ 
cal  data  provide  a  powerful  tool  when  combined  with  a  test 
capability.  The  potential  exists  to  provide  100%  fault 
detection/fault  isolation  capability." 

Paragraph  40.2.6.2a:  Add  the  following: 

"(7)  Testability  and  fault-tolerant  design." 

Paragraph  40.3.1.8:  After  item  "e.",  add  the  following  item 
"f"  and  reletter  the  present  "f"  and  "g"  accordingly. 

"f.  Diagnostic  Capability  Demonstrations.  In  accordance 
with  MIL-STD-2165,  the  testability  demonstration  is  to  be 
performed  as  part  of  the  maintainability  demonstrations. 
Thus  the  ground  rules  for  the  MD  should  address  all  elements 
that  make  up  the  diagnostic  capability  (i.  e.,  testing, 

technical  information,  personnel,  and  training)  that  are  re¬ 
quired  to  meet  fault  detection  and  isolation  requirements. 
This  includes  continuity,  consistency,  and  compatibility  of 
the  diagnostic  capability  between  maintenance  levels." 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 

REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-2165 

TITLE:  TESTABILITY  PROGRAM  FOR  ELECTRONIC 

SYSTEMS  AND  EQUIPMENTS 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  ATTS/MNTY 

PREPARING  NAVSEA,  WASHINGTON,  D.  C. 

ACTIVITY:  PAUL  GROSS,  202-692-2035 

DOCUMENT  PURPOSE: 

The  standard  provides  uniform  procedures  and  methods 
for  establishing  a  testability  program,  for  assessing  tes¬ 
tability  in  designs,  and  for  integration  of  testability  into 
the  acquisition  process  for  electronic  systems  and  equip¬ 
ment.  Expansion  of  the  document's  scope  is  proposed  to 
cover  the  entire  weapon  system's  diagnostic  capability,  in¬ 
cluding  design  characteristics  of  the  weapon  system  itself 
and  associated  testing,  technical  information,  and  personnel 
and  training. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

This  standard  will  be  the  prime  programmatic  document 
for  controlling  the  acquisition  of  a  weapon  system's  diag¬ 
nostic  capability. 

GENERAL  RECOMMENDATIONS: 

It  is  proposed  that  this  standard  be  expanded  to  cover 
the  design  characteristics  of  the  weapon  system  itself  (e. 
g.,  fault  tolerant  design)  and  other  diagnostic  elements 
that  constitute  the  diagnostic  capability.  The  standard 
will  no  longer  be  limited  to  just  electronics,  but  will  ad- 
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dress  the  entire  weapon  system.  Therefore,  the  title  should 
be  changed  to  "Testability/Diagnostic  Program  Requirements 
for  Systems,  Subsystems,  and  Equipments."  The  intent  is  to 
strengthen  the  design  and  integration  portion  of  the  present 
standard,  while  lessening  the  requirements  for  program 
monitoring  and  control,  and  test  and  evaluation  by  utilizing 
other  existing  standards  to  perform  these  functions.  In  ad¬ 
dition,  when  M I L -STD- 4 15  is  revised  to  address  design 
criteria  for  the  entire  diagnostic  capability,  it  must  be 
referenced  at  appropriate  places  in  MIL-STD-2165 . 

SPECIFIC  MODIFICATIONS: 

Substantial  modifications  follow,  many  times  requiring 
a  complete  rewrite  of  sections  and  paragraphs. 

1.  SCOPE 

1.1  Purpose 

This  standard  provides  uniform  procedures  and  methods 
for  establishing  a  program  for  acquiring  a  weapon  system's 
diagnostic  capability,  for  assessing  this  capability  and 
design,  and  for  integration  of  all  of  the  elements  which 
constitute  this  capability  into  the  acquisition  process. 

1.2  Application 

This  standard  is  applicable  to  the  development  of  all 
types  of  weapon  systems,  subsystems,  and  equipments  for  the 
Department  of  Defense.  Appropriate  tasks  of  this  standard 
are  to  be  applied  during  the  Concept  Exploration  Phase, 
Demonstration  and  Validation  Phase,  Full-Scale  Development 
Phase,  and  Production/Deployment  Phase  of  the  system  ac¬ 
quisition  process. 

3.  DEFINITIONS  AND  ACRONYMS 

3.1  Definitions 

The  definitions  included  in  MIL-STE\-1309  and  MIL-STD- 
721  shall  apply.  In  addition,  the  definitions  of  Appendix  C 
are  applicable. 
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4.  GENERAL  REQUIREMENTS 

4.1  Scope  of  Diagnostic  Program 

This  standard  is  intended  to  impose  and  facilitate  in¬ 
terdisciplinary  efforts  required  to  develop  an  adequate 
diagnostic  capability  for  weapon  systems  and  equipments. 
The  program  scope  includes: 

(a)  Support  of,  and  integration  with,  reliability  and 
maintainability  design,  including  retirements  for 
deferred  maintenance,  performance  monitoring,  and 
corrective  maintenance  action  at  all  levels  of 
maintenance. 

(b)  Support  of  integrated  logistic  support  require¬ 
ments,  including  the  support  and  test  equipment, 
human  engineering,  technical  publications,  train¬ 
ing  and  training  equipment,  and  safety  elements. 

(c)  Support  of,  and  integration  with,  design  engin¬ 
eering  requirements,  including  hierarchical 
development  of  testability  designs  from  piece/part 
to  the  system. 

4.2  Testability  Program  Requirements 

A  diagnostic  program  shall  be  established  which  ac¬ 
complishes  the  following  general  requirements: 

(a)  Establishment  of  program  planning  requirements 

(b)  Establishment  of  a  sufficient,  achievable,  and 
affordable  weapon  system  diagnostic  capability 

(c)  Integration  of  the  diagnostic  elements  that  con¬ 
stitute  the  entire  weapon  system's  diagnostic 
capabil ity 

(d)  Evaluation  of  the  extent  to  which  the  design 
meets  the  diagnostic  requirements 

(e)  Inclusion  of  diagnostics  in  the  program  review 
process . 
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4.3  Application  of  the  Requirements 

Detailed  requirements  described  in  this  standard  are  to 
be  selectively  applied  and  tailored,  as  required  and  as  ap¬ 
propriate,  to  particular  systems  and  equipment  acquisition 
programs.  Maximum  use  has  been  made  of  reliability,  main¬ 
tainability,  logistics,  and  human  engineering  program  tasks 
as  vehicles  for  implementing  the  diagnostic  program.  Appen¬ 
dix  A  provides  the  rationale  and  guidance  for  the  selection 
and  tailoring  of  diagnostic  program  tasks. 

5.  DETAILED  REQUIREMENTS 
5.1  Task  Descriptions 

Individual  task  requirements  are  provided  for  the  es¬ 
tablishment  of  a  weapon  system  diagnostic  capability 
program.  The  tasks  are  categorized  as  follows: 

TASK  SECTION  100.  PROGRAM  MONITORING  AND  CONTROL 

Task  101,  Diagnostic  Capability  Program  Planning 
Task  102,  Diagnostic  Reviews 

Task  103,  Diagnostic  Data  Collection  and  Analysis 
Planning 

TASK  SECTION  200.  DESIGN  AND  ANALYSIS 
Task  201,  Diagnostic  Requirements 

Task  202,  Testability/Diagnostic  Preliminary  Design  and 
Analysis 

Task  203,  Testability/Diagnostic  Detail  Design  and 
Analysis 

TASK  SECTION  300.  TEST  AND  EVALUATION 

Task  301,  Diagnostic  Inputs  to  Maintainability 
Demonstrations 

6.  NOTES 

When  this  standard  is  used  in  an  acquisition,  the  data 
identified  below  shall  be  deliverable  only  when  specified  on 
the  DD  Form  1423,  Contract  Data  Requirement  List  (CDRL)  . 
When  the  DD  Form  1423  is  not  used  and  Defense  Acquisition 
Regulation  7-1 04 . 9 (n) (2 )  is  cited,  the  data  identified  below 
shall  be  delivered  in  accordance  with  requirements  specified 


4-38 


APPLICABLE  DATA  ITEM 

TASK  DATA  REQUIREMENT  DESCRIPTION  (DID) 


101 


Systems  Engineering  UDI-E-23974  (1)  (2) 


102  Diagnostic  Reviews  DI-E-5423  (1) 

201,202,203  Testability  Analysis 

Report  DI-T-7199 

301  Maintainability 

Demonstration 

Test  Plan  DI-R-7112 

Maintainability 

Demonstration  DI-R-7113 


(1) 

Equivalent 

approved 

DI 

D  may 

be 

used  . 

(2) 

This  DID  should  be 

ta  i 

lored 

to 

r equ i re 

inclusion  of 

Diagnostic 

Capabil i 

ty 

Prog  r 

am 

Pi ann i ng 

in  the  SEMP. 
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TASK  101 


DIAGNOSTIC  CAPABILITY  PROGRAM  PLANNING 


101.1  PURPOSE.  To  plan  for  a  program  which  will  identify 
and  integrate  all  design  management  tasks  required  to  ac¬ 
quire  a  diagnostic  capability  for  a  weapon  system. 

101.2  TASK  DESCRIPTION 

101.2.1  Identify  a  single  organizational  element  within  the 
performing  activity  which  has  overall  responsibility  and 
authority  for  implementation  of  the  program.  Establish 
analyses  and  data  interfaces  among  the  organizational  ele¬ 
ments  responsible  for  each  of  the  elements  of  the  diagnostic 
capability  and  other  related  elements. 

101.2.2  Develop  a  process  by  which  diagnostic  requirements 
are  integrated  with  other  design  requirements  and  dissemi¬ 
nated  to  design  personnel  and  subcontractors.  Establish 
controls  for  assuring  that  each  subcontractor's  diagnostic 
practices  are  consistent  with  overall  system  or  equipment 
requirements . 

101.2.3  Identify  diagnostic  design  guides  and  analysis 
models  and  procedures  to  be  imposed  upon  the  design  process. 
Plan  for  the  review,  verification,  and  utilization  of  diag¬ 
nostic  data  submissions. 

101.2.4  Develop  a  Diagnostic  Capability  Program  Plan  which 
describes  how  the  program  will  be  conducted.  The  program 
plan  shall  be  included  as  part  of  the  Systems  Engineering 
Management  Plan.  The  plan  describes  the  time  phasing  of 
each  task  included  in  the  contractual  requirements  and  its 
relationship  to  other  tasks.  Diagnostic  issues  which  relate 
to  reliability,  maintainability,  logistics,  human  engineer¬ 
ing,  safety,  etc.,  should  be  addressed  in  those  plans. 

101.3  TASK  INPUT 

101.3.1  Identification  of  each  diagnostic  task  which  is  re¬ 
quired  to  be  performed  as  part  of  the  program.* 


I 


r 


101.3.2  Identification  of  the  time  period  over  which  each 
task  is  to  be  conducted.* 

101.3.3  Identification  of  approval  procedures  for  plan 
updates  .* 

101.3.4  Identification  of  deliverable  data  items.* 

101.4  TASK  OUTPUT 

101.4.1  The  program  plan  is  to  be  a  part  of  the  System  En¬ 
gineering  Management  Plan. 


f 


*  To  be  specified  by  the  requiring  authority. 
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TASK  102 


DIAGNOSTIC  REVIEWS 


102. 

ing 


1  PURPOSE, 
activity  to 


To  establish  a  requirement  for  the  perform- 
(1)  provide  for  an  official  review  of  diag¬ 
nostic  design  information  in  a  timely  and  controlled  manner, 
and  (2)  conduct  in-process  diagnostic  design  reviews  at 
specified  dates  to  ensure  that  the  program  is  proceeding  in 
accordance  with  the  contract  requirements  and  program  plans. 


102.2  TASK  DESCRIPTION 


102.2.1  Include  the  formal  reviews  and  assessment  of  the 
diagnostic  program  as  an  integral  part  of  each  system 
program  review  (e,  g..  System  Design  Review,  Preliminary 
Design  Review,  Critical  Design  Review,  etc.)  specified  by 
the  contract.  Reviews  shall  cover  all  pertinent  aspects  of 
the  diagnostic  program,  such  as: 

(a)  Status  and  results  of  diagnostic-related  tasks 

(b)  Documentation  of  task  results  in  the  testability 
analysis  report 

(c)  Diagnostics-related  requirements  in  speci f icati ons 

(d)  Diagnostic  design,  cost,  or  schedule  problems. 

Use  MIL-STD-1521,  Technical  Reviews  and  Audits  for  Sys¬ 
tems,  Equipments,  and  Computer  Programs  as  guidance  for  con¬ 
ducting  these  formal  reviews. 

102.2.2  Conduct  and  document  diagnostic  design  reviews  with 
performing  activity  personnel  and  with  subcontractors  and 
suppliers.  Coordinate  and  conduct  diagnostic  reviews  in 
conjunction  with  reliability,  maintainability,  human  en¬ 
gineering,  and  logistic  support  reviews,  whenever  possible. 
Utilize  MIL-STD-1521  and  program  review  criteria  contained 
in  MIL-STDs  470,  785,  and  1388-1  as  guidance. 
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102.3  TASK  INPUT 

102.3.1  Identification  of  amount  of  time  to  be  devoted  to 
the  diagnostic  program  at  each  formal  review  and  the  level 
of  technical  detail  to  be  provided.* 

102.3.2  Identification  of  level  of  participation  desired  by 
the  requiring  authority  in  internal  and  subcontractor  diag¬ 
nostic  design  reviews.* 

102.4  TASK  OUTPUT 


102.4.1 

integral 

(102.2.1) 


Documented  results  of  diagnostic  assessment  as  an 
part  of  system  program  review  documentation. 
(See  MIL-STD- 1521) 


102.4.2 
includ ing 


Documented  results  of 
action  items  pending. 


diagnostic 

(102.2.2) 


design 


reviews , 


To  be  specified  by  the  requiring  authority. 
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TASK  103 


DIAGNOSTIC  DATA  COLLECTION  AND  ANALYSIS  PLANNING 


103.1  PURPOSE.  To  establish  a  method  for  identifying  and 
tracking  diagnostic-related  problems  during  system  design, 
production,  and  deployment  and  identifying  corrective  ac¬ 
tions. 

103.2  TASK  DESCRIPTION 

103.2.1  Develop  a  plan  for  the  analysis  of  test  results  to 
determine  if  BIT  hardware  and  software,  ATE  hardware  and 
software,  maintenance  documentation,  and  manning  are  meeting 
specifications  in  terms  of  fault  detection,  fault  resolu¬ 
tion,  fault  detection  times,  and  fault  isolation  times. 

103.2.2  Develop  a  plan  for  the  analysis  of  maintenance  ac¬ 
tions  for  the  fielded  system  to  determine  if  BIT  hardware 
and  software,  ATE  hardware  and  software,  maintenance 
documentation,  and  manning  are  meeting  specifications  in 
terms  of  fault  detection,  fault  resolution,  false  indica¬ 
tions,  fault  detection  times,  and  fault  isolation  times. 

103.2.3  Define  data  collection  requirements  to  meet  the 
needs  of  the  testability  analysis.  The  data  collected  shall 
include  a  description  of  relevant  operational  anomalies  and 
maintenance  actions.  Data  collection  shall  be  integrated 
with  similar  data  collection  procedures,  such  as  those  for 
reliability  and  maintainability  and  logistic  support 
analysis  and  shall  be  compatible  with  specified  data  systems 
in  use  by  the  military  user  organization. 

103.3  TASK  INPUT 

103.3.1  Identification  of  field  or  depot  test  equipment 
(either  government-furnished  equipment  or  contractor- 
furnished  equipment)  to  be  available  for  development, 
production,  and  deployment  testing.* 


103.3.2  Identification  of  existing  data  collection  systems 
in  use  by  the  using  command.* 

103.3.3  Relationship  of  Task  103  to  Task  104  of  MIL-STD-785 
and  Task  104  of  MIL-STD-470 .* 

103.4  TASK  OUTPUT 

Utilize  data  systems  established  under  MIL-STD-785  and 
MIL-STD-470. 


*  To  be  specified  by  the  requiring  authority. 
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TASK  201 


DIAGNOSTIC  REQUIREMENTS 


201.1  PURPOSE.  To  (1)  recommend  system  diagnostic  require¬ 

ments  which  best  achieve  mission  performance,  availability 
and  supportability  requirements,  and  (2)  allocate  those  re¬ 
quirements  to  subsystems  and  items  addressing  all  diagnostic 
functions  (i.  e.,  test  equipment,  technical  publications, 

personnel,  training). 

201.2  TASK  DESCRIPTION 

201.2.1  Establish  overall  diagnostic  design  objectives, 
goals,  thresholds,  and  constraints  which  support  mission  re¬ 
quirements  and  operational  constraints  in  support  of  the 
logistic  support  analysis  process  of  MIL-STD-1388-1A  and  the 
system  engineering  process.  Inputs  to  these  requirements 
include: 

(a)  Translation  of  weapon  system  mission  and  perfor¬ 
mance  requirements  into  diagnostic  requirements 
which  support  the  mission  scenario. 

(b)  Establishment  of  requirements  which  allow  for 
diagnostic  growth  as  design  proceeds  through  the 
weapon  system  acquisition  phases. 

(c)  Identification  of  diagnostics-related  constraints 
driven  by  operational  constraints  of  the  system. 

(d)  Identification  of  technology  advancements  which 
can  be  exploited  in  system  development  and  diag¬ 
nostic  element  development  and  which  have  the 
potential  for  increasing  diagnostic  effectiveness; 
reducing  the  requirement  for  maintenance;  reducing 
test  equipment,  technical  manuals  and  manpower, 
and  skill-level  requirements;  reducing  diagnostics 
costs;  or  enhancing  system  availability. 

(e)  Identification  of  existing  and  planned  diagnostic 

resources  (e.  g.,  family  of  testers,  maintenance 

aids)  which  have  potential  benefits.  Identify 
resource  limitations. 

(f)  Identification  of  diagnostics  problems  on  similar 
systems  which  should  be  avoided. 
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201.2.2  Define  what  constitutes  a  system  failure  and  estab¬ 
lish  deferred  maintenance,  performance  monitoring,  embedded 
diagnostic,  and  external  diagnostic  objectives  for  the  new 
system  at  the  system  and  subsystem  levels.  Identify  the 
risks  and  uncertainties  involved  in  achieving  the  objectives 
established . 

201.2.3  Establish  BIT,  test  equipment,  technical  informa¬ 
tion,  and  maintenance  manpower  and  skill-level  constraints 
for  the  new  system  for  inclusion  in  system  specifications  or 
other  requirements  documents.  These  constraints  shall  in¬ 
clude  both  quantitative  and  qualitative  constraints. 

201.2.4  Evaluate  alternative  diagnostic  concepts  to  include 
varying  degrees  of  BIT,  manual  and  automatic  testing,  tech¬ 
nical  information  format  and  delivery  systems,  personnel  and 
training,  along  with  deferred,  preventive,  and  scheduled 
maintenance  concepts,  and  identify  the  selected  concept. 
The  evaluation  shall  include: 

(a)  A  determination  of  the  sensitivity  of  system 
mission  performance  and  readiness  parameters  to 
variations  in  key  diagnostic  element  parameters. 

(b)  A  determination  of  the  sensitivity  of  life  cycle 
costs  to  variations  in  diagnostic  element 
parameters . 

(c)  An  estimation  of  the  manpower  and  personnel  impli¬ 
cations  of  alternative  diagnostic  concepts  in 
terms  of  direct  maintenance  manhours  per  operating 
hour,  job  classification,  skill  levels,  and  ex¬ 
perience  required  at  each  level  of  maintenance. 

(d)  An  estimation  of  the  risk  associated  with  each 
concept . 

201.2.5  Establish  embedded  diagnostic  requirements,  perfor¬ 
mance  requirements,  and  requirements  for  monitors  and  sen¬ 
sors  at  the  system  and  subsystem  level.  These  requirements 
include  specific  numeric  performance  requirements  imposed  by 
the  requiring  authority.  Other  requirements  shall  be  based, 
in  part,  on: 
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(a)  Maximum  allowable  time  between  the  occurrence  of  a 
failure  condition  and  the  detection  of  the  failure 
for  each  mission  function. 

(b)  Maximum  allowable  occurrence  of  system  downtime 
due  to  erroneous  failure  indications  (BIT  false 
alarms) . 

(c)  Maximum  allowable  downtime  due  to  corrective  main¬ 
tenance  actions  at  the  Organizational  level. 

(d)  Minimum  life  cycle  costs. 

201.2.6  Recommend  system  diagnostic  design  provisions,  BIT 
and  testability  requirements,  manual  and  automatic  test, 
technical  information,  and  personnel  and  training  require¬ 
ments  for  inclusion  in  system  specifications.  Appendix  A, 
Figure  5,  provides  guidance  on  requirements  to  be  specified. 

201.2.7  Allocate  embedded  diagnostics  and  testability  re¬ 
quirements  to  configuration  item  specifications,  based  upon 
reliability  and  criticality  considerations. 

201.2.8  Prepare  external  diagnostic  element  specifications. 

201.3  TASK  INPUT 

201.3.1  Supportabil ity  analysis  data  in  accordance  with 
MIL-STD-1388-1A  or  other  method  approved  by  the  requiring 
authority. 

201.3.2  Reliability  and  maintainability  analysis  and  re¬ 
quirements,  such  as  from  Task  203  of  MIL-STD-785  and  Task 
205  of  MIL-STD-470. 

201.3.3  Specific  numeric  diagnostic  and  testability 
requirements .* 

201.3.4  Human  Engineering  Analysis  and  Requirements,  such 
as  from  MIL-H-46885,  paragraph  3.2.1. 


*  To  be  specified  by  the  requiring  authority. 
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201.4  TASK  OUTPUT 


201.4.1  Diagnostic  data  required  for  suppo r tab i 1 i ty 
analysis.  (201.2.1  through  201.2.4) 

201.4.2  Description  of  selected  diagnostic  concept  and 
trade-off  methodology,  evaluation  criteria,  models  used,  and 
analysis  results,  documented  in  accordance  with  DI-T-7199. 

201.4.3  Recommended  diagnostic  and  testability  requirements 
for  system  specification.  (201.2.3  and  201.2.6) 

201.4.4  Recommended  diagnostic  and  testability  requirements 
for  each  conf iguration  item  specification.  (201.2.7) 

201.4.5  Specifications  for  external  diagnostic  elements. 


TASK  202 


TESTABILITY/DIAGNOSTICS  PRELIMINARY  DESIGN  AND  ANALYSIS 


Incorporate  the  following  modifications: 

201.1  PURPOSE.  To  incorporate  testability/diagnostic 
design  practices  into  design  of  the  system  or  equipment 
early  in  the  design  phase  and  to  assist  the  extent  to  which 
testability  is  incorporated. 

Renumber  202.2.5  to  202.2.6  and  insert  the  following: 

"202.2.5  Determine  the  impact  of  the  diagnostic  parametric 
values  on  reliability,  maintainability,  technical  informa¬ 
tion,  manpower,  training,  and  external  diagnostics  hardware 
and  software." 

202.3.4  Human  Engineering  Analysis,  such  as  from  MIL-H- 
46855,  paragraph  3. 2. 1.3. 

Renumber  202.4.5  to  202.4.6  and  insert  the  following: 

"202.4.5  Preliminary  design  for  diagnostic-related 
tradeoffs  and  optimization,  in  accordance  with  DI-T-7199. 
Develop  and  document  viable  alternative  diagnostic  element 
concepts  (off-line  test  equipment,  technical  information)." 
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TASK  203 


TESTABILITY/DIAGNOSTICS  DETAIL  DESIGN  AND  ANALYSIS 

203.1  PURPOSE.  To  incorporate  the  diagnostic  capability 
into  the  design  of  a  system,  equipment,  or  support  equipment 
which  will  satisfy  diagnostic  performance  requirements  and 
to  predict  the  level  of  diagnostic  effectiveness  which  will 
be  achieved  for  the  weapon  system  or  equipment. 

203.2  TASK  DESCRIPTION 

Make  the  following  changes  in  Task  203.2: 

203.2.4  through  203.2.6.  Replace  the  word  "BIT"  with  the 
words  "embedded  diagnostics." 

203.2.8  Prepare  test  requirements  document,  in  accordance 
with  MIL-STD-1 51 9 . 

203.2.9  Develop  and  define  off-line  test  equipment  require¬ 
ments.  (Input  to  Task  207,  MIL-STD-470) 

203.2.10  Define  test  program  set  requirements. 

203.2.11  Determine  diagnostic  technical  information  needs 
and  method  of  delivery  of  this  information.  (Input  to  Task 
207,  MIL-STD-470) 

203.2.12  Determine  manpower,  training,  and  training  equip¬ 
ment  requirements  and  develop  required  hardware  and 
software.  (Input  to  Task  207,  MIL-STD-470  and  Phase  I, 
MIL-STD-1379) 

203.2.13  Incorporate  testability  and  diagnostic  corrective 
design  actions,  as  determined  by  the  maintainability 
demonstration  results  and  initial  testing. 

203.2.14  Identical  to  present  203.2.9,  except  replace  the 
word  "BIT"  with  the  words  "embedded  diagnostics." 

203.3.3  Replace  "BIT  specifications."  with  "Diagnostic 
specifications . " 

203.3.6  Human  Engineering  Equipment  Design,  such  as  from 
MIL -H -46855,  paragraph  3.2.2. 
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T* 


203.4.1  Replace  with  "System  or  item  design  which  meets 
diagnostic  and  maintainability  requirements  (203.2.1; 
203.2.4;  203.2.5;  203.2.8;  203.2.9;  203.2.10;  203.2.11; 
203.2.12;  203.2.13)  ." 
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TASK  301 


DIAGNOSTIC  INPUTS  TO  MAINTAINABILITY  DEMONSTRATIONS 


Paragraph  301.1  and  301.2.1:  Replace  "testability  require¬ 

ments"  with  "diagnostic  requirements". 

Paragraph  301.2.2  and  301.2.3:  Replace  "testability"  with 

"diagnostics" . 

Delete  paragraphs  301.4.1  and  301.4.2  and  replace  with: 
"301.4  TASK  OUTPUT.  Outputs  shall  be  in  accordance  with 
MIL-STD-470,  Task  301." 
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APPENDIX  A 


DIAGNOSTIC  PROGRAM  APPLICATION  GUIDANCE 


Major  changes  will  have  to  be  made  in  Appendix  A  to 
reflect  the  changes  in  the  basic  military  standard.  Major 
modifications  include: 

o  Revision  to  the  samples  specifications,  reflecting 
what  is  contained  in  the  National  Security  In¬ 
dustrial  Association's  "Guidelines  for  Preparation 
of  Diagnostic  Requirements." 

o  Specific  diagnostic  inputs  to  the  reliability,  main¬ 
tainability,  and  logistic  program  plans. 

o  Revision  of  the  program  flow  diagrams  and  the  task 
application  matrix. 

The  content  of  Appendix  A  must  be  compatible  with  the 
Testability  Analysis  Handbook,  which  is  presently  under 
preparation.  Much  of  the  material  presently  contained  in 
this  appendix  will  be  addressed  in  this  handbook.  Thus  it 
is  important  that  duplication  be  minimized  when  Appendix  A 
is  revised.  Topics  not  covered  by  the  handbook  should  be 
included,  and  reference  needs  to  be  made  to  material  in  the 
handbook . 


APPENDIX  C 


GLOSSARY 

Add  additional  terms,  such  as: 
o  Diagnostic  Capability 
o  Diagnostic  Element 
o  integrated  Diagnostics 
o  Embedded  Diagnostics 
o  External  Diagnostics 
o  Maintenance  Aid 
o  Expert  Diagnostic  System 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 


REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-1629A 


TITLE:  PROCEDURES  FOR  PERFORMING  A  FAILURE 

MODE,  EFFECTS,  AND  CRITICALITY 
ANALYSIS 


EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  RELIABILITY 


PREPARING  NAVAIR  5112,  WASHINGTON,  D.  C. 

ACTIVITY:  JOHN  COOK,  NAEC ,  201-323-7458 

DOCUMENT  PURPOSE: 


To  establish  requirements  and  procedures  for  performing 
a  failure  mode,  effects,  and  criticality  analysis  (FMECA)  to 
systematically  evaluate  and  document,  by  item  failure  mode 
analysis,  the  potential  impact  of  each  functional  or 
hardware  failure  on  mission  success,  personnel  and  system 
safety,  system  performance,  maintainability,  and  maintenance 
requirements.  Each  potential  failure  is  ranked  by  the 
severity  of  its  effect  in  order  that  appropriate  corrective 
actions  may  be  taken  to  eliminate  or  control  the  high-risk 
i terns . 


TESTABILITY/DIAGNOSTIC  IMPACT: 

It  is  through  the  FMECA  that  the  diagnostic  parametric 
values  can  be  specified  and  evaluated  for  percentage  levels, 
ambiguity  size,  impact  on  reliability  and  maintainability, 
impact  on  training  and  personnel,  and  impact  on  system  per¬ 
formance  and  system  safety. 
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GENERAL  RECOMMENDATIONS: 

At  present,  no  significant  changes  in  this  standard  are 
required  from  a  diagnostic  perspective.  However,  Task  103, 
FMECA--Maintainability  Information,  is  loosely  coupled  to 
the  diagnostic  design  process.  At  present,  this  coupling  is 
addressed  in  MIL-STD-2165 .  Direct  coupling  is  required  be¬ 
tween  Task  103  and  the  test  logic  and  test  point  selection 
process.  This  can  best  be  accomplished  by  the  proposed 
revision  to  MIL-STD-415,  which  will  address  this  subject. 

SPECIFIC  MODIFICATIONS: 

Delete  all  references  to  MIL-STD-2080A  (AS),  Main¬ 
tenance  Engineering  Planning  and  Analysis  for  Aeronautical 
Systems,  Subsystems,  Equipment,  and  Support  Equipment,  since 
this  standard  is  in  the  process  of  being  canceled. 

Section  2,  Reference  Documents,  and  paragraph  4.3.6 
should  include  a  reference  to  MIL-STD-2165. 

Paragraph  10.3  should  be  revised  to  insert  the  words 
"testability,  diagnostics"  after  the  word 
"maintainability,". 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  2 

REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-H-46855B 

TITLE:  HUMAN  ENGINEERING  REQUIREMENTS  FOR 

MILITARY  SYSTEMS,  EQUIPMENT,  AND 
FACILITIES 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  HFAC 

PREPARING  ARMY  AMSMI-EDS,  REDSTONE  ARSENAL,  AL 

ACTIVITY:  KATHY  BROOKS,  205-876-0663 

DOCUMENT  PURPOSE: 

This  specification  establishes  and  defines  the  require¬ 
ments  for  applying  human  engineering  to  the  development  and 
acquisition  of  military  systems,  equipment,  and  facilities. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

This  standard  provides  the  human  engineering  input  to 
the  Integrated  Diagnostics  process,  thus  it  will  have  a  sig¬ 
nificant  input  to  the  revised  MIL-STD-2165. 

GENERAL  RECOMMENDATIONS: 

Significant  modifications  to  this  document  are  required 
to  facilitate  its  use.  These  recommendations  follow. 

o  At  present,  permission  has  been  received  to  retain 
this  document  as  a  military  specification.  The  pur¬ 
pose  and  content  of  the  document  is  identical  to 
that  of  other  programmatic  standards  covered  in  this 
report,  including  the  system  engineering  standard, 
MIL-STD-499.  Utilization  of  this  document  in  the 
form  of  a  specification  is  difficult.  Such  things 
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as  references  to  appropriate  DIDs  are  not  included 
in  specifications.  it  is  recommended  that  this 
document  be  reissued  as  a  standard. 

o  The  format  for  this  specification  uses  an 

application  matrix  of  text  modification  for  im¬ 
plementing  the  requirements  over  the  various  weapon 
system  acquisition  phases.  This  format  is  not  con¬ 
ducive  to  easy  use.  It  is  recommended  that  the  for¬ 
mat  used  by  other  programmatic  standards  be  applied. 

o  The  application  matrix  contains  guidance  on  when 

each  human  engineering  function/task  should  be  per¬ 
formed  in  relation  to  the  weapon  system  acquisition 
cycle.  Many  of  the  tasks  appear  to  be  "out  of  sync" 
with  the  requirements  in  other  programmatic  stan¬ 
dards,  such  as  MIL-STD-1388-1A  and  MIL-STD-499.  An 
example  is  the  requirement  to  perform  workload 
analysis  in  the  Concept  Exploration  Phase  of  weapon 
system  acquisition. 

o  Although  the  specification  does  clearly  recognize  a 
need  for  an  allocation  of  functions  between  human, 
machine,  and  software,  the  required  allocation 
described  is  not  based  on  a  mission-driven  analysis. 

o  MIL-STD-2165  should  be  revised  to  reference  inputs 
from  MIL-H-46855  in  relation  to  human  engineering. 

SPECIFIC  MODIFICATIONS: 

Revise  paragraph  3.2.1,  as  follows: 

"3.2.1  Analysis .  Mission  analyses  shall  be  conducted  to 
include:  (1)  a  review  of  man/machine  problems  on  similar 

systems  and  (2)  a  translation  of  mission  requirements  into 
system  functions  and  subfunctions.  The  analyses  shall  in¬ 
clude  application  of  human  engineering  techniques  as  fol¬ 
lows  : 

(a)  Previous  Man/Machine  Problems.  The  operational 

and  maintenance  histories  of  previous,  similar 
systems  shall  be  reviewed  to  identify  man/machine 
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problems  to  be  avoided  in  the  system  under 
development.  Types  of  problems  to  look  for  con¬ 
cern  human  responsibilities,  manning,  procedures, 
documentation,  skill  levels,  and  training. 

(b)  Mission  and  Function  Analyses.  The  contractor 

shall  identify  and  describe  system 
functions/subfunctions  without  regard  to  the  means 
of  implementation  (e.  g.,  without  describing 
whether  man,  machine/software,  or  some  combination 
will  perform  the  functions) .  The  basis  for  iden¬ 
tifying  and  describing  these  functions  shall  be 
the  results  of  the  analysis  of  3. 2. 1.1  and  a 
review  of  mission  data  (e.  g.,  system  operational 
requirements,  modes  of  operation,  usage,  and  sup¬ 
port,  mission  objectives,  and  operating 
environments) . 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.: 
REQUIREMENT  TITLE: 


DOCUMENT  NUMBER: 

TITLE: 

EXISTING:  X 

STANDARDIZATION 
AREA: 

PREPARING 
ACTIVITY: 

DOCUMENT  PURPOSE: 

This  standard  establishes  requirements  to  be  applied 
during  the  development  and  acquisition  of  mission-critical 
computer  system  (MCCS)  software,  as  defined  in  DoD  Directive 
5000.29.  This  standard  may  also  be  applied  to  non-MCCS 
software  development  and  acquisition. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  requirements  for  software  development  will  impact 
the  development  of  diagnostic  software  embedded  in  weapon 
systems,  subsystems,  and  in  off-line  ATE. 

GENERAL  RECOMMENDATION: 

This  standard  defined  the  process  requirements  for  the 
development  of  computer  software  configuration  items  (CSCI) 
and  computer  software  components  (CSC)  .  The  performance  re¬ 
quirements  for  CSCI  and  CSC  are  contained  in  various  types 
of  speci f icat ions ,  as  delineated  in  correlating  DIDs.  For 
example,  the  system/segment  specification  is  defined  in  DI- 
CMAN-80008.  This  same  DID  is  called  out  in  MIL-STD-490A. 


2 

DESCRIBING  VARIOUS  TESTABILITY/ 
DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOD-STD-2167 

DEFENSE  SYSTEM  SOFTWARE  DEVELOPMENT 
PLANNED: 


MCCR 

SPAWAR,  WASHINGTON,  D.  C. 
202-692-3535 
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The  comments  made  for  MIL-STD-490A  DID  changes  also  apply  to 
MIL-STD-2167.  Namely,  the  System/Subsystem  Specification 
DID  (DI -CMAN-80008 )  should  be  modified  to  include  a 
paragraph  called  "Diagnostics". 

SPECIFIC  MODIFICATIONS: 

No  changes  are  recommended  for  the  Software  Requirements 
and  Software  Product  Specifications. 


DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.  2: 

REQUIREMENT  TITLE:  DESCRIBING  VARIOUS  TESTABILITY/ 

DIAGNOSTIC  TASKS  WHICH  MUST  BE  UNDER¬ 
TAKEN  DURING  EACH  PHASE  OF  WEAPON 
SYSTEM  ACQUISITION 

DOCUMENT  NUMBER:  MIL-STD-13 7 9B 

TITLE:  CONTRACT  TRAINING  PROGRAMS 

EXISTING:  X  PLANNED: 

STANDARDIZATION  69GP 

AREA: 

PREPARING  NAVAIR,  51122,  WASHINGTON,  D.  C. 

ACTIVITY:  CAROLYN  PARKER,  305-646-5912  OR  5592 

NAVAL  TRAINING  EQUIPMENT  CENTER, 

ORLANDO,  FL 

DOCUMENT  PURPOSE: 

This  standard  establishes  the  requirements  for  prepar¬ 
ing,  validating,  verifying,  conducting,  and  revising  train¬ 
ing  programs,  which  are  required  to  qualify  military  and 
civilian  technicians,  instructors,  or  other  personnel  to 
operate,  program,  maintain,  repair,  overhaul,  and  instruct 
on  the  system/equipment. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  standard  deals  with  all  types  of  training,  includ¬ 
ing  maintenance  training. 

GENERAL  RECOMMENDATIONS: 

The  standard  should  be  revised  to  address  on-the-job 
training  in  relation  to  electronic  delivery  devices 
(maintenance  aids) . 

SPECIFICATIONS  MODIFICATIONS: 

Paragraph  5.1.3:  After  the  word  "tools",  add  ",  maintenance 
aids.". 
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Replace  paragraph  5.5.12  with  the  following  paragraph: 
"5.5.12  On-The-Job  Training  (OJT)  Material.  On-the-job 
training  (OJT)  material  shall  be  prepared  to  be  utilized  by 
the  using  activity  as  its  primary  vehicle  in  providing  an 
effective  maintenance  capability  in  support  of  the  deployed 
system/equipment.  This  material  can  take  the  form  of  an 
on-the-job  handbook  or  an  electronic  delivery  device  and  can 
be  used  to  enhance  and  freshen  technician  training  and 
facilitate  training  of  less  experienced  crew  members.  As 
such,  the  training  material  must  be  developed  to  address  the 
needs  of  both  experienced  and  inexperienced  technicians. 
Modifications  to  this  OJT  maintenance  training  material 
shall  be  enhanced  to  take  advantage  of  knowledge  gained 
through  operational  experience  or  during  presentations  of 
training  courses." 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.: 
REQUIREMENT  TITLE: 
DOCUMENT  NUMBER: 
TITLE: 

EXISTING:  X 


3 

DESIGNING  THE  DIAGNOSTIC  CAPABILITY 
MIL-STD-454J 

STANDARD  GENERAL  REQUIREMENTS  FOR 
ELECTRONIC  EQUIPMENT 

PLANNED: 


STANDARDIZATION 

AREA:  GDRQ 


PREPARING 

ACTIVITY: 


HQ,  AFSC/PLEQ ,  ANDREWS  AFB ,  WASHINGTON, 
D.  C. 

CAPT.  SYLVESTER,  301-981-2751 

HQ,  ASD/ENES,  WRIGHT-PATTERSON  AFB,  OH 

ROGER  FAUST,  513-255-6295 


DOCUMENT  PURPOSE: 


The  standard  covers  the  common  requirements  to  be  used 
in  military  specifications  for  electronic  equipment.  Each 
requirement  is  intended  to  cover  some  discipline  in  the 
design  of  equipment,  such  as  the  procedure,  a  process,  or 
the  selection  and  application  of  parts  and  materials. 

TESTABILITY/DIAGNOSTICS  IMPACT: 

Reliability,  maintainability,  and  human  engineering  are 
requirements  that  are  included  in  this  standard.  These  dis¬ 
ciplines  do  not  adequately  address  testability  and  diagnos¬ 
tics.  For  these  types  of  engineering  disciplines,  the  data 
sheets  stress  that  this  standard  does  not  establish  require¬ 
ments  and  must  not  be  referenced  in  contractual  documents. 
Thus  these  three  requirement  examples  offer  direction  on 
what  must  be  considered  in  preparing  contractual  documents. 

GENERAL  RECOMMENDATION: 

It  appears  that  the  above  three  (3)  requirement  data 
sheets  were  included  in  this  standard  to  alert  users  that 
there  were  requirements  for  reliability,  maintainability, 
and  human  engineering.  Since  these  requirements  cannot  be 
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referenced  in  contractual  documents,  their  inclusion  is 
somewhat  superficial.  In  fact,  the  NSIA  Integrated  Diagnos¬ 
tics  Group  recommended  that  Integrated  Diagnostics  not  be 
included  in  the  standard.  They  felt  that  the  standard  was 
of  benefit  only  to  the  specification  preparer  and  not  the 
user.  In  addition,  they  felt  that  a  requirement  for 
automatic  test  equipment  not  be  included  but,  rather,  ad¬ 
dressed  in  MIL-STD-41 5 . 

SPECIFIC  MODIFICATIONS: 

Although  it  is  somewhat  superficial,  it  is  recommended 
that  the  attached  Requirement  No.  76,  which  addresses  In¬ 
tegrated  Diagnostics,  be  included  in  this  standard  to  alert 
users  on  how  to  address  this  type  of  process. 
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REQUIREMENT  76 


INTEGRATED  DIAGNOSTICS 


1.  Purpose :  This  requirement  establishes  a  design  process 

for  integrating  all  elements  which  constitute  a  weapon 
system's  diagnostic  capability-  IT  DOES  NOT  ESTABLISH  RE¬ 
QUIREMENTS,  AND  MUST  NOT  BE  REFERENCED  IN  CONTRACTUAL  DOCU¬ 
MENTS.  Engineering  analyses,  quantitative  requirements, 
design  analysis,  and  demonstration  and  maturation  require¬ 
ments  must  be  specified  in  the  contract  or  system/equ i pment 
specification,  as  appropriate. 

2 .  Documents  Applicable  to  Requirement  76 : 

MIL-STD-2165  --  Testability  Program  for  Electronic 

Systems  and  Equipments 

MIL-STD-4  71  --  Maintainability  Verification/Demon¬ 

stration/Evaluation 

3 .  Integrated  Diagnostics  Process: 

Integrated  Diagnostics  is  defined  as  a  structured 
process  which  maximizes  the  effectiveness  of  diagnostics  by 
integrating  pertinent  elements,  such  as  testability, 
automatic  and  manual  testing,  training,  maintenance  aiding, 
and  technical  information  as  a  means  for  providing  a  cost- 
effective  capability  to  detect  and  unambiguously  isolate  all 
faults  known  or  expected  to  occur  in  weapon  systems  and 
equipment  and  to  satisfy  weapon  system  mission  requirements. 

This  emphasis  on  the  design  and  acquisition  of  the 
diagnostic  capability  is  required  because  this  capability 
transcends  a  multitude  of  other  design  disciplines  (e.  g., 

reliability,  maintainability,  logistics  support,  human 
engineering) ,  and  thus  the  capability  tends  to  become  frac¬ 
tionated.  The  major  implementing  document  is  MIL-STD-2165. 
However,  because  it  is  a  mu  1 1 id i sc i pi i ned  process,  reference 
will  probably  be  required  to  portions  of  other  military 
standards  and  specifications  (e.  g.,  MIL-STD-785,  M I L -STD- 

13  8  8  -  1  ,  M I L-STD - 470,  MIL-STD-499,  and  M I L -H - 4 6 8 5 5 ) . 


DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO. : 
REQUIREMENT  TITLE: 
DOCUMENT  NUMBER: 
TITLE: 

EXISTING:  X 


3 

DESIGNING  THE  DIAGNOSTIC  CAPABILITY 
MIL-STD-4 1 5D 

DESIGN  CRITERIA  FOR  TEST  PROVISIONS 
FOR  ELECTRONIC  SYSTEMS  AND  ASSOCIATED 
EQUIPMENT 

PLANNED: 


STANDARDIZATION 

AREA:  MISCELLANEOUS 


PREPARING  ASD/ENES,  WRIGHT-PATTERSON  AFB ,  DAYTON 

ACTIVITY:  OH 

MARY  COURY,  513-255-6295 


DOCUMENT  PURPOSE: 

This  standard  establishes  design  criteria  for  test 
provisions  that  permit  the  functional  and  static  parameters 
of  electronic  systems  and  associated  equipment  to  be 
monitored,  evaluated,  or  isolated. 


TESTABILITY/DIAGNOSTIC  IMPACT: 

The  last  revision  to  this  standard  was  in  October  1971, 
and  thus  the  design  criteria  is  outdated  and  needs  to  be 
rewr i tten . 


GENERAL  RECOMMENDATIONS: 

The  standard  addresses  older  technologies,  such  as  ex¬ 
ternal  test  receptacles  for  connecting  prime  equipment  to 
automatic  monitoring  equipment.  A  complete  rewrite  of  the 
standard  is  required  to  address  new  technologies,  system  ar¬ 
chitectures,  and  the  prime  system's  entire  diagnostic 
capabi 1 i ty . 
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SPECIFIC  RECOMMENDATIONS 


The  following  is  an  outline  of  the  content  and  thrust 
of  the  new  standard. 


TITLE  —  Design  Criteria  for  Diagnostic  Provisions 
for  Systems  and  Equipments 


SECTION  1  —  SCOPE 

The  scope  and  purpose  of  the  standard  should  be  ex¬ 
panded  to  address  design  criteria  for  the  entire  prime 
system's  diagnostic  capability. 

SECTION  3  —  DEFINITIONS 

Some  of  the  definitions  are  obsolete,  such  as  automatic 
monitoring  equipment.  The  entire  list  of  definitions  should 
be  screened  and  appropriate  MIL-STD-721  and  MIL-STD-1309 
definitions  incorporated. 

SECTION  4  —  GENERAL  REQUIREMENTS 

The  entire  section  on  general  requirements  is  aimed  at 
the  analysis  to  determine  test  provisions  that  are  required 
for  operation  and  maintenance.  This  type  of  information  is 
presently  required  by  a  number  of  programmatic  standards, 
foremost  of  which  is  MIL-STD-2165  (both  the  present  MI  L- 
STD-2165  and  the  proposed  revised  standard).  Reference 
should  be  made  to  these  standards,  while  alerting  the  desig¬ 
ner  that  he  must  take  into  consideration  the  various  diag¬ 
nostic  elements  which  must  be  considered  (e.  g.,  performance 
monitoring,  f aul t-tolerant  systems,  electronic  delivery  of 
technical  information,  and  on-the-job  training  aids) .  In 
addition,  the  designer  should  be  reminded  that  this  covers 
the  entire  weapon  system,  including  diagnostics  for  mechani¬ 
cal  and  structural  items. 

SECTION  5  —  DETAIL  REQUIREMENTS 

Paragraph  5.2.1,  General  Design  Considerations,  should 
be  expanded  to  include  the  following  subjects: 

o  The  concept  of  vertical  testab i 1 i ty-- f r om  factory 
to  field 
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o  The  requirement  for  a  cohesive  diagnostic  design 

process  to  address  diagnostic  and  test  program 
development/generation,  treated  as  a  single  design 
process  applicable  to  all  levels  of  maintenance  (e. 
g.,  interfaces  among  MIL-STDs  756,  1629,  1519,  1345, 
2077) 

o  Support  strategies  for  VLSI/VHSIC 

o  Failure  reporting  and  logging  of  field  performance 
of  the  weapon  system's  diagnostic  capability, 
treated  as  a  single  entity,  addressing  all  level  of 
maintenance. 

Paragraph  5.2.2,  Automatic  Checkout  and  Automatic 
Monitoring  Capabilities,  needs  to  be  completely  rewritten  to 
address  off-line  ATE  in  lieu  of  on-line  ATE.  Some  of  the 
design  criteria  which  should  be  addressed  include: 

o  The  utilization  of  IEEE  ATLAS  716  as  the  test 
language 

o  Control  of  the  interface  between  the  UUT  and  the 
ATE,  including  criteria  covering  the  design  of  the 
interface  device 

o  Guidance  on  the  appropriate  use  of  general-purpose 
ATE,  as  opposed  to  special  purpose. 

Paragraph  5.2.3  deals  with  BIT  capability.  This 
paragraph  needs  to  be  expanded  to  cover  such  design  criteria 
f  or : 

o  Performance  monitoring/status  monitoring/in-flight 
monitoring  and  recording,  including  performance 
degradation 

o  Performance  monitoring  in  relation  to  f ault-tolerant 
design 

o  Performance  monitoring  for  mechanical  and  structural 
items,  including  sensor  design  criteria  and  time- 
stress  measurement  devices 

o  Test  measurement  bus  structure 

o  Smart  BIT  (expert  systems  technology) 
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o  Delete  the  requirement  for  the  interpretation  of  BIT 
output  by  low-skilled  personnel 

o  Reference  should  be  made  to  the  Joint  Service 

Built-In  Test  Design  Guide  and  the  NBS  Sensor  Hand¬ 
book  for  Test,  Monitoring,  Diagnostic,  and  Control 
System  Applications  to  Military  Vehicles  and 
Machinery. 

o  Review  contents  of  Task  104,  MIL-STD-2084  (AS)  for 
possible  inclusion,  since  this  standard  establishes 
criteria  for  the  design  and  application  of  BIT. 

Paragraph  5.2.4,  Test  Points,  should  be  revised  to  ad¬ 
dress  design  criteria  for  test  point  selection.  The 
criteria  delineated  in  Section  6  of  MIL-STD-1326  (Navy) 
should  be  considered  for  this  expansion,  including  the  use 
of  paragraph  3.0  in  the  appendix  to  this  standard,  as  an  ex¬ 
ample  of  optimum  test  point  selection.  The  design  criteria 
contained  in  MIL-STD-2084  (AS),  Task  105,  should  also  be 
reviewed  for  inclusion. 

Add  a  new  paragraph:  "5.2.5  Electronic  Delivery  of 

Technical  Information."  This  paragraph  should  address  the 
following  design  criteria. 

o  Types  of  information  (e.  g.,  procedural,  historical, 
systematic) 

o  Maintenance  aiding,  including  the  application  uf 
expert  system  technology 

o  Training  aids,  embedded  in  the  prime  system  and 

external  to  the  prime  system,  utilized  for  on-the- 
job  training 

o  Compatibility  of  technical  information  used  for  both 
maintenance  aiding  and  training 

o  Application  of  standards  addressing  the  authoring  of 
technical  information  (e.  g.,  LISP)  and  interchange 
of  technical  information  (e.  g.,  CALS  standards, 
test  buses,  BIT) 

o  The  interface  with  common  data  bases. 

o  The  configuration  management  of  expert  system  soft¬ 
ware,  which  may  differ  from  weapon  system  to  weapon 
system  of  the  same  type. 
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Paragraph  5.3,  Human  Engineering  Requirements,  should 
be  expanded  to  address  design  criteria  and  standards  util¬ 
ized  in  transferring  technical  information  from  the  machine 
(e.  g.,  ATE,  maintenance  aids,  training  aids)  to  the  tech¬ 
nician.  This  includes  addressing  comprehensibility,  for¬ 
mats,  ready  access  to  information,  etc. 

Section  6  needs  revision  to  modify  the  ordering  data 
and  data  preparation  instructions  to  conform  to  the  addi¬ 
tional  requirements  contained  in  Section  5. 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  3 


REQUIREMENT  TITLE: 


DESIGNING  THE  DIAGNOSTIC  CAPABILITY 


DOCUMENT  NUMBER: 


MIL-STD-1345B  (NAVY ) /MIL-STD-1519 
(USAF) /MI L-STD-15 19  (UPDATE)  (MATE) 


TITLE: 


PREPARATION  OP  TEST  REQUIREMENTS 
DOCUMENT 


EXISTING:  X 


PLANNED: 


STANDARDIZATION 

AREA:  MISC 


PREPARING  NAVSEA  55Z3,  WASHINGTON,  D.  C. 

ACTIVITY:  JEAN  HARMON,  202-692-0160 

ASD/ENES,  WRIGHT-PATTERSON  AFB ,  DAYTON, 
OH 

MARY  COURY,  513-255-6295 

DOCUMENT  PURPOSE: 


To  establish  the  requirements  for  the  preparation  and 
control/acceptance  of  the  Test  Requirements  Document  (TRD) 
used  in  specifying  testing  requirements  for  electronic 
system/subsystems,  units,  assemblies/subassemblies  referred 
to  as  units  under  test  (UUTs) . 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  TRD  provides  the  UUT  data  needed  to  specify  =>nd 
design  the  off-line  diagnostics  (i.  e.,  ATE/TPS  FD/FI 
levels)  . 

GENERAL  RECOMMENDATIONS: 

Three  MIL-STDs  currently  delineate  requirements  for  a 
TRD:  MIL-STDs  1345  (Navy),  1519  (USAF),  and  1519 
(Updated/MATE) .  It  is  recommended  that  the  three  documents 
be  combined  into  one  standard,  incorporating  the  elements  of 
all  three,  as  shown  in  the  specific  modifications  below. 
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SPECIFIC  MODIFICATIONS: 


The  above  three  standards  all  have  the  same  title, 
"Test  Requirements  Document,  Preparation  of."  The  amount  of 
detail  required  in  each  TRD  increases  by  placing  the  stan¬ 
dards  in  the  following  order. 

MIL-STD-1519  requires  the  minimum,  MIL-STD-1345B  in¬ 
creases  MIL-STD-1519 ,  and  MIL-STD-1519  (Updated)  increases 
the  requirements  of  MIL-STD-1345B .  A  comparison  of  TRD  re¬ 
quirements  that  impact  testability/diagnostics  is  shown 
below. 


TRD  REQUIREMENT 

MIL- 

STD-1519 

MIL-STD-1345B 

MIL-STD-1519 

(UPDATED) 

1.  ATPG  Requirements 

Not 

addressed 

Stated 

Not  addressed 

2.  ATLAS  Requirements 

3.  Testing  of  Built-In 

Not 

addressed 

Stated 

Stated 

Test  (BIT) 

4.  Testability  Analy¬ 

Not 

addressed 

Stated 

Stated 

sis  Report 

5.  Interface  to  Prime 

Not 

addressed 

Not  addressed 

Stated 

System  Life  Cycle 

Not 

addressed 

Not  addressed 

Defined 

6.  CAD/CAE  Data  ReqmtS, 

7.  Failure  Rate  and 
Fault  Isolation 
Ambiguity  Group 

Not 

addressed 

Not  addressed 

Not  addressed 

Summary 

Not 

addressed 

Tables  provided 

Table  provided 

MIL-STD-1519  (Updated)  is  included  in  MATE  Guide  5, 
Volume  3,  Part  6  data.  It  was  prepared  using  both  MIL-STD- 
1519  and  MlL-STD-1345  as  references. 

From  the  above  analysis,  it  can  be  seen  that  MIL-STD- 
1519  does  not  address  ATPG  data  and  does  not  contain  re¬ 
quirements  for  CAD/CAE  data.  MIL-STD-1519  (Updated)  has 
been  formatted  to  provide  Part  A,  Minimum  Essential  Data  Re¬ 
quirements,  at  the  completion  of  the  UUT  PDR,  Part  3; 
Developmental  Test  Requirements  and  Testability  Data 
Analysis,  at  the  completion  of  the  UUT  CDR;  and  Part  C, 
Production  Base  Line  Data,  at  the  completion  of  the  UUT  FCA 
and  again  at  the  completion  of  the  UUT  PCA. 
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It  is  recommended  that  MIL-STD-1519  (Updated)  be 
revised  to  include  ATPG  and  CAD/CAE  data  requirements  and, 
after  the  revision  is  completed,  be  identified  as  the  sole 
test  requirements  document  standard  by  canceling  MIL-STD- 
1519  and  MI L-STD-1 34 5B . 

Recommended  CAD/CAE  data  requirements  that  should  be 
included  in  the  TRD  Standard: 

"X.X.l  CAD/CAE  Data  Requirements.  Contractors  who  design 
UUT  with  CAD/CAM  systems  shall  provide  the  following  data: 

a.  Identification  of  the  CAD/CAE  system  used 

b.  Description  of  test  strategy  included  as  part  of 
the  UUT  design 

c.  Failure  modes,  effects,  and  criticality  analysis 

d.  Predicted  fault  detection  and  fault  isolation 
levels 

e.  Media  on  which  test  vectors  are  stored 

f.  Identification  of  UUT  built-in  test 

g.  Descriptions  of  UUT  interface  characteristics 

h.  Copies  of  all  the  original  "source  listings"  of  any 
test  sequence 

i.  Data  postprocessing  requirements. 

X.X.2  Automatic  Test  Program  Generation  (ATPG)  System 
Information. 

Incorporate  paragraph  5.5.3  of  MIL-STD-1345B  into  the 
revised  MIL-STD-1519  (Updated)." 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  3 

REQUIREMENT  TITLE:  DESIGNING  THE  DIAGNOSTIC  CAPABILITY 
DOCUMENT  NUMBER:  MIL-STD-2077  (NAVY) 

TITLE:  GENERAL  REQUIREMENTS  FOR  TEST  PROGRAM 

SETS 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  MISCELLANEOUS 

PREPARING  NAVAIR,  51122,  WASHINGTON,  D  C. 

ACTIVITY:  WALTER  CORNETZ,  NAEC ,  201-323-7489 

DOCUMENT  PURPOSE: 

This  document  establishes  a  standard  for  design, 
development,  documentation,  configuration  management, 
validation,  verification,  quality  assurance  and  preparation 
for  delivery  of  Test  Program  Sets  (TPS). 

TESTABILITY/DIAGNOSTICS  IMPACT: 

The  requirements  for  TPS  development  stated  in  this 
standard  provide  a  method  for  developing  TPSs  that  meet 
specified  off-equipment  diagnostic  capability  parametric 
values  of  fault  detection/fault  isolation. 

GENERAL  RECOMMENDATIONS: 

This  standard  has  been  developed  for  the  Naval  Air  Sys¬ 
tems  Command,  the  Space  and  Naval  Warfare  Systems  Command, 
and  the  Naval  Sea  Systems  Command.  It  uses  terms  and  states 
requirements  that  are  peculiar  to  Navy  utilization  of  TPSs. 
The  Foreword  of  the  standard  requires  that  careful  attention 
to  tailoring  of  the  requirements  be  accomplished  to  avoid 
duplication  of  data  and  to  ensure  that  TPSs  provide  useful 
information  to  serve  their  intended  function. 
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It  is  recommended  that  a  "tailoring"  section  be  in¬ 
cluded  in  the  standard  to  serve  as  guidance  to  selectively 
apply  and  tailor  the  MIL-STD  requirements  for  Air  Force  and 
Army  applications. 

SPECIFIC  MODIFICATIONS: 

Add  the  following  sentence  to  the  Foreword  on  page  iii: 
"Specific  tailoring  guidance  for  Army  and  Air  Force  users  of 
the  standard  is  provided  in  Appendix  D." 

Page  12,  paragraph  5. 1.7. 9:  Change  second  sentence  to  read: 
"All  test  programs  shall  be  designed  to  provide  maximum  per¬ 
cent  fault  detection  consistent  with  (JUT  testability,  ATE 
design,  and  reasonable  cost  targets;  minimum  fault  detection 
shall  be  specified  as  a  percentage  of  total  detectable 
faults." 

Add  Appendix  D,  as  follows: 

"APPENDIX  D 

APPLICATION  GUIDE  FOR  TAILORING  MI L-STD -20 7 7 ( A ) 


10  SCOPE 

10.1  General .  This  appendix  sets  forth  guidance  for  the 
application  of  this  standard  for  Air  Force  and  Army  require¬ 
ments  . 

10.2  Purpose .  The  guidelines  contained  herein  selectively 
provide  for  the  conversion  of  Navy-peculiar  TPS  requirements 
to  Air  Force  and  Army  applications. 

20  CONSIDERATIONS  FOR  TAILORING 

20.1  Air  Force.  The  Air  Force  has  developed  and  institu¬ 
tionalized  the  MATE  System.  The  MATE  System  contains 
detailed  guidance,  tools,  and  models  developed  specifically 
for  TPS  specification,  design,  and  support.  The  procuring 
activity  shall  use  the  MATE  System  TPS  products  for  guidance 
in  the  generation  of  a  TPS  SOW  and  specification  that  con¬ 
forms  to  the  requirements  of  this  standard  and  the  MATE  Sys¬ 
tem. 
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20.2  Army.  The  Army  Materiel  Command,  through  the  Product 
Manager--TPS,  has  developed  and  distributed  an  AMC  TPS  Pro¬ 
cedures  Manual.  This  TPS  Procedures  Manual  provides  the 
guidance  and  tools  necessary  for  the  acquisition,  design, 
and  support  of  Army  TPSs.  Army  Program  Managers  shall  use 
the  AMC  TPS  Procedures  Manual  for  guidance  in  the  generation 
of  Army  TPS  SOWS  and  specifications.  " 


DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.: 
REQUIREMENT  TITLE: 
DOCUMENT  NUMBER: 
TITLE: 

EXISTING:  X 

STANDARDIZATION 

AREA: 


3 

DESIGNING  THE  DIAGNOSTIC  CAPABILITY 

JOINT  SERVICE  BUILT-IN  TEST  DESIGN  GUIDE 
PLANNED: 

ATTS 


PREPARING  NAVSEA,  WASHINGTON,  D.  C. 

ACTIVITY:  PAUL  GROSS,  202-692-2035 

DOCUMENT  PURPOSE: 


The  Joint  Service  Built-In  Test  (BIT)  Design  Guide 
provides  detailed  guidelines  on  the  implementation  of  BIT. 
It  provides  information  on  development  of  BIT  strategies, 
BIT  tradeoffs,  and  BIT  design  techniques.  It  is  a  guidance 
document  only  and  contains  no  requirements. 

TESTABILITY/DIAGNOSTICS  IMPACT: 

BIT  is  a  key  diagnostic  element.  Proper  design  and  im¬ 
plementation  of  BIT  is  crucial  to  an  effective  diagnostics 
capability.  Therefore,  guidance  on  BIT  design  is  very  im¬ 
portant  and  the  BIT  Design  Guide  serves  an  important  func¬ 
tion  in  diagnostics  implementation. 

GENERAL  RECOMMENDATIONS: 


Although  the  BIT  Design  Guide  is  over  six  years  old,  it 
contains  very  useful  and  relevant  information  on  BIT  design. 
It  was  recently  updated,  with  a  section  on  component  BIT. 
The  guide  should  be  updated  to  include  additional  BIT  topics 
to  assure  that  all  aspects  of  BIT  are  addressed.  Much  of 
the  useful  information  in  the  guide  is  contained  within  sec¬ 
tions  which  are  oriented  toward  specific  examples  of  BIT  im¬ 
plementation.  Unless  the  user  reads  the  guide  cover-to- 
cover,  he  is  not  likely  to  find  certain  key  guidelines. 
Therefore,  an  index  of  the  information  in  the  guide  would  be 
extremely  useful. 
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I 

SPECIFIC  MODIFICATIONS: 

f 

The  BIT  Design  Guide  is  the  only  guide  oriented  toward 
1  BIT.  Several  other  very  good  design  guides  exist  and  are 

readily  available,  such  as  the  MATE  Avionics  Testability 
Design  Guide  and  the  RADC  Testability  Notebook.  These 
should  be  referenced  in  the  BIT  Design  Guide,  so  that  a  user 
needing  additional  information  can  be  informed  where  and  how 
to  find  it. 

The  information  contained  in  the  BIT  Design  Guide  is 
t  excellent,  but  not  totally  complete.  The  BIT  Design  Guide 

should  be  expanded  to  include  sections  on  BIT  Prognostics, 
BIT  Software,  and  Analog  BIT.  The  BIT  Requirements  Analysis 
section  should  be  expanded  and  given  an  integrated  diagnos¬ 
tics  flavor.  The  BIT  approaches  section  should  be  updated 
to  include  test  and  maintenance  bus  techniques  at  the  system 
level  and  integration  aspects  of  BIT.  A  section  should  be 
added  on  BIT  Evaluation. 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  3 

REQUIREMENT  TITLE:  DESIGNING  THE  DIAGNOSTIC  CAPABILITY 
DOCUMENT  NUMBER:  MIL-HDBK-338 


TITLE: 


ELECTRONIC  RELIABILITY  DESIGN  HANDBOOK 
VOLS.  1  AND  2 


STANDARDIZATION 

AREA:  RELIABILITY 


PREPARING  RADC,  ROME,  NY 

ACTIVITY:  CHARLES  MESSENGER,  315-330-3766 

DOCUMENT  PURPOSE: 


This  handbook  is  an  update  and  extensive  revision  of 
the  Reliability  Design  Handbook,  published  in  1976  by  the 
Reliability  Analysis  Center  under  contract  to  RADC.  The 
handbook  contains  pertinent  and  practical  guidelines  for  use 
by  design  engineers,  reliability  engineers,  and  managers  to 
design,  produce,  and  deploy  reliable  and  maintainable 
military  electronic  eguipment/systems  at  minimum  life  cycle 
cost. 


TESTABILITY/DIAGNOSTIC  IMPACT: 

Although  the  title  of  the  document  implies  a  strict 
reliability  focus,  the  handbook  describes  a  comprehensive 
methodology  covering  many  aspects  of  electronic  system 
design  engineering  and  cost  analysis  as  they  relate  to  the 
design  acquisition,  and  deployment  of  DoD  equipment/systems. 
The  sections  on  Failure  Modeling  (5.2.3),  Reliability  Pre¬ 
diction  Techniques  (6.4),  Transient  and  Overstress  Protec¬ 
tion  (7.4.4),  Parameter  Degradation  and  Analysis  (7.4.5), 
Redundancy  (7.5),  Failure  Modes  and  Effects  Analysis  (7.8), 
Fault  Tree  Analysis  (7.9),  and  Software  Reliability  (9.0) 
are  strongly  linked  to  testability  and  diagnostics. 

GENERAL  RECOMMENDATIONS: 

The  sections  described  above  should  be  expanded  to  ac¬ 
commodate  current  state-of-the-art  (S.O.A.)  techniques  and 
theory  applicable  to  these  diagnostics-related  technology 
areas . 
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SPECIFIC  MODIFICATIONS: 


Section  5.2.3  (Failure  Modeling)  is  essential  to  the 
diagnostic  design  process,  but  its  treatment  in  MIL-HDBK-338 
addresses  mathematical  (i.  e.,  failure  rate),  as  opposed  to 
the  actual  physical  models  of  both  "good"  and  "bad"  devices. 
Although  validated  failure  rate  models  are  essential  to  the 
development  of  reliability  prediction  techniques,  this  ap¬ 
proach  provides  a  level  of  abstraction  far  removed  from 
reality.  From  a  diagnostics  point  of  view,  the  reliability 
discipline,  in  conjunction  with  the  electrical  engineering 
discipline,  must  provide  generic  model  representations  of 
fault-free  devices  (i.  e.,  VHDL,  gate  level,  circuit  level, 
etc.).  "Faulty"  device  models  also  need  to  be  developed, 
which  emulate,  from  an  electrical  point  of  view,  known 
device  failure  modes.  This  section  of  the  handbook  must 
come  to  grips  with,  and  address,  this  difficult,  but  impor¬ 
tant,  concept  of  physical  fault  modeling  in  order  for  the 
handbook  to  have  applicability  to  broader  diagnostic  arenas. 

Section  6.4  (Reliability  Prediction  Techniques)  should 
be  augmented  to  encompass  how  the  reliability  discipline  can 
be  utilized  to  predict  test  effectiveness  as  part  of  the 
weapon  system  design  process.  In  particular,  the  utiliza¬ 
tion  of  reliability-based  data  to  predict  testability- 
related  Figures  of  Merit,  such  as  Fraction  of  Faults 
Detected  (FFD) ,  Fraction  of  False  Alarms  (FFA) ,  Fraction  of 
False  Status  Indications  (FFSI),  Test  Thoroughness  (TT) ,  and 
Fraction  of  Faults  Isolated  (FFI)  should  be  discussed  in  a 
new  section,  titled:  "Mathematical  Models  for  Testability 
Prediction." 

Section  7.4.4  (Transient  and  Overstress  Protection) 
should  be  expanded  to  include  transmission  systems — both 
eiectr ical/electronic  (i.  e.,  cable,  bus  systems)  and  fiber 
optics.  Computerized  transient  analysis  modeling/prediction 
techniques  should  also  be  addressed. 

Section  7.4.5  (Parameter  Degradation  and  Analysis) 
should  be  modified  and  expanded  to  provide  more  in-depth 
criteria  and  methodology  on  how  to  perform  nominal,  worst 
case,  Monte  Carlo,  and  parameter  variation  analysis.  These 
techniques  should  be  demonstrated  utilizing  computer-aided 
design  tools,  such  as  SPICE.  The  result  of  the  methodology 
should  be  a  design  and  analysis  approach  which  enables  the 
designer  to  derive  technically  sound  performance  limits  for 
the  subject  unit  under  test.  Utilizing  these  performance 
limits,  the  derivation  of  reliable  test  limits,  utilizing 
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UUT  performance  limits  and  ATE/BIT 
stimulus/measurement/switching  performance  uncertainties 
should  be  demonstrated. 

This  approach  will  enable  designers,  in  essence,  to 
design  more  reliable  equipment,  by  virtue  of  designing  for 
realistic  testing  tolerance  across  all  levels  of  equipment 
indenture  (i.  e..  Factory,  Depot,  Intermediate,  Organiza¬ 
tional  Levels)  and  eliminating/minimizing  the  Retest  Okay 
(RTOK)  problem. 

The  sections  on  redundancy,  in  paragraph  7.5 
(Redundancy)  and  as  amplified  in  Appendix  A  (Redundancy  Con¬ 
siderations  in  Design),  are  excellent.  From  a  diagnostic 
viewpoint,  these  sections  should  be  updated  and  expanded  to 
provide  more  detailed  examples  of  various  modern  BIT  tech¬ 
niques,  utilized  in  concert  with  redundancy,  and  their 
resulting  impact  on  overall  system  reliability.  In  addi¬ 
tion,  Section  7.5  (Redundancy)  should  be  expanded  to  address 
reliable  design  techniques  to  fault  detect  and  transmit  data 
pertaining  to  faults  occurring  in  systems  employing  redun¬ 
dancy  techniques.  The  issue  here  is  that,  unless  one  knows 
whether  a  fault  has  indeed  occurred,  one  cannot  predict  the 
reliability  of  a  device  at  a  particular  instant  in  time. 
The  upcoming  Tri-Service  Testability  Analysis  Handbook  and 
BIT  Design  Guide  should  be  utilized  as  input  to  this  update. 

Section  7.8  (Failure  Modes  and  Effects  Analysis)  should 
be  augmented  to  provide  an  in-depth  and  up-to-date  discus¬ 
sion  of  computer-assisted  failure  mode  and  effects  analysis. 
The  present  discussion  in  the  handbook  (paragraph  7.8.5, 
Computer  analysis)  is  both  terse  and  obsolete.  Modern  cir¬ 
cuit  analysis  programs  (i.  e.,  analog  and  digital)  and  fault 
models  currently  available  to  simulate  failures  in 
electronic  circuits  and  to  automatically  ascertain  their  ef¬ 
fects  should  be  discussed  in  detail  in  this  section,  along 
with  appropriate  examples,  to  demonstrate  the  key  concepts 
presented . 

The  utilization  of  FMEA  techniques  to  analyze  "soft 
fault"  conditions  currently  being  experienced  in  modern-day 
weapon  systems  should  also  be  discussed  in  depth. 

Section  7.9  (Fault  Tree  Analysis)  should  be  rewritten, 
utilizing  modern-day  logic  modeling  techniques  (i.  e.,  de¬ 
pendency  modeling)  to  demonstrate  how  a  failure  mode  at  one 
level  of  equipment  indenture  can  produce  a  failure  at  a 
higher  level  in  the  system.  Utilization  of  modern,  cur- 
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rently  available  computerized  tools,  such  as  LOGMOD  or  the 
Navy's  Weapon  System  Testability  Analyzer  (WSTA) ,  should  be 
applied  within  this  section  to  demonstrate  the  concept  of 
automated  fault  tree  analysis. 

Section  9.0  (Software  Reliability)  should  be  augmented 
to  address  modern-day  thinking  with  respect  to  software 
validation/verification  techniques.  Specifically,  Section 
9.8.4  (program  Checking  and  Testing)  should  be  expanded  to 
incorporate  new  debugging  and  integration  testing  techniques 
to  ensure  fault-free  software  systems. 

In  summary,  this  document  would  serve  as  an  excellent 
vehicle  for  addressing  the  specific  design  aspects  of  design 
for  testability/diagnostics  as  it  relates  to  reliability  and 
maintainability.  It  is  probably  one  of  the  better,  if  not 
the  best,  design  handbooks  available  within  DoD. 


DOCUMENT  ANALYSIS  SHEET 


REQUIREMENT  NO.:  3 

REQUIREMENT  TITLE:  DESIGNING  THE  DIAGNOSTIC  CAPABILITY 

DOCUMENT  NUMBER: 

TITLE:  SENSOR  HANDBOOK  FOR  TEST,  MONITORING, 

DIAGNOSTIC  AND  CONTROL  SYSTEM  APPLICA¬ 
TIONS  TO  MILITARY  VEHICLES  AND 
MACHINERY 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  ATTS 

PREPARING 

ACTIVITY:  NBS,  WASHINGTON,  D.  C. 

DOCUMENT  PURPOSE: 

The  Sensor  Handbook  is  intended  as  a  guide  for  those 
who  design,  specify,  use,  and  test  military  automatic  test 
equipment  containing  sensors.  The  handbook  addresses 
measurands  and  principles  of  measurement,  data  acquisition, 
sensor  calibration  and  testing,  environmental  considera¬ 
tions,  stability,  durability,  reliability,  and  error  assess¬ 
ment.  The  handbook  is  addressed  to  the  general  engineer, 
system  designer,  or  manager  with  an  engineering  background. 
It  does  not  provide  the  highly  detailed  technical  informa¬ 
tion  needed  by  a  design  engineer,  although  ample  references 
are  included  for  further  study. 

TECHNOLOGY/DIAGNOSTIC  IMPACT: 

The  primary  application  of  the  handbook  is  for  BIT  ap¬ 
plications  which  require  various  sensor  technologies  to 
detect/measure  various  parameters  such  as  pressure,  motion, 
flow,  temperature,  force  and  torque,  level,  humidity  mois¬ 
ture,  etc.,  which  are  being  experienced  by  a  weapon  system 
or  elements  of  a  weapon  system. 
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GENERAL  RECOMMENDATIONS: 


There  should  be  a  modification  and  update  of  the  hand¬ 
book  to  reflect  sensor  technologies  and  design  techniques  to 
enable  the  design  of  "Smart  BIT"  by  a  weapon  system  desig¬ 
ner  . 

SPECIFIC  MODIFICATIONS: 

Based  upon  BIT  experience  to  date  on  major  programs  (i. 
e.,  B-1B ,  F-16,  F-18,  etc.),  identify  sensor  requirements 
for  "Smart  BIT."  Develop  a  generic  "Smart  BIT"  sensor 
detection,  data  acquisition,  and  data  analysis  scheme  which 
can  be  tailored  to  specific  weapon  systems.  In  particular, 
focus  on  the  primary  culprits  of  BIT  false  alarms,  such  as 
electrical  transients  and  vibration  in  the  development  of 
this  "Smart  BIT"  architectural  design.  Emphasis  should  be 
placed  on  the  specification  and  utilization  of  generic  sen¬ 
sors  by  design  engineers  in  the  context  of  a  generalized 
diagnostic  data  acquisition  and  data  analysis  system. 
Generic  software  schemes  to  sample,  filter,  log,  and  analyze 
BIT  sensor  data  should  also  be  provided.  An  outline  of  the 
proposed  addition  to  the  handbook  is  provided  below: 


"SENSOR  APPLICATIONS  FOR  SMART  BIT 
1.0  INTRODUCTION 

1.1  Need  for  Smart  BIT  (False  Alarm  Problem) 

1.2  Sensor  Role  in  Smart  BIT 

2.0  SMART  BIT  MODEL 

2.1  Model  Overview 


DAM 

STORAGE 
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2.2 

Function  of 

Threshold 

Detector 

2.3 

Determining 

Pass/Fail 

Limits 

2.4 

BIT  Processor 

2.4.1  System  Data  Requirements 

2.4.2  Data  Storage  Requirements 

2.4.3  Criteria  for  Alarm  Generation 

3.0  SMART  BIT  SENSOR  REQUIREMENTS 


3.1 

Minimum,  But  Essential,  Sensor  Technology 
Requirements 

3.1.1 

Vibration/Shock 

3.1.2 

Noise 

3.1.3 

Transients 

3.1.4 

Voltage/Current  Sensing 

3.1.5 

Temperature/Humidity  Sensing 

3.1.6 

Angular  Displacement 

3.1.7 

Pressure  Displacement 

3.1.8 

Motion 

3.1.9 

Force/Torque 

3.2 

Sensor 

Operational  Attributes 

3.2.1 

Calibration 

3.2.2 

Fault  tolerance/Redundancy 

3.3 

Sensor 

Application  Attributes 

3.3.1 

Packaging  (Hardware) 

3.3.2 

Programmability  (Adaptable  Software) 

4.0  SMART  BIT  SYSTEM  ARCHITECTURE  UTILIZING  SENSOR 
TECHNOLOGY 

4.1  Generic  (Adaptable)  System  Architecture 

4.2  Sensor  Interface  Requirements 

4.3  Data  Acquisition  Requirements 

4.4  Operational  Software  Requirements 

4.4.1  Minimizing  False  Alarms  While  Maintaining 
Safety 

4.4.2  Filtering  Out  Non-Solid  Failures 

4. 4. 2.1  Retry  Failing  Operation  "n"  times 

4. 4. 2. 2  Declare  Solid  Failure  Only  if 
"n"  No-Gos 

4. 4. 2. 3  Replacement  of  Corrupted  Data 

4.4.3  Log  Intermittent  Failures/Sensor  Data 
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4.4.4  Improve  Probability  of  Sensor  Fault 
Detection 

4. 4. 4.1  Cycle  instructions 

4. 4. 4. 2  Cycle  Programs 

4. 4. 4. 3  Cycle  BIT  and  Acquire  Sensor  Data 
Under  Different  System  Operational 
Scenarios 

4. 4. 4. 4  Log  "Soft"  Failure  History/Sensor 
Data 

4. 4. 4. 5  Determining  the  Need  for 
"Adaptive"  Test  Limits 

4.5  Criteria  and  Application  Guidelines  for  Applying 
Sensor  to  Smart  BIT  Applications" 


In  addition,  the  handbook  should  be  coordinated  and 
issued  as  a  military  handbook. 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.: 
REQUIREMENT  TITLE: 
DOCUMENT  NUMBER: 


4 

CONDUCTING  DESIGN  REVIEWS 
MIL-STD-1521B  (USAF) 


TITLE:  TECHNICAL  REVIEWS  AND  AUDITS  FOR 

SYSTEMS,  EQUIPMENT,  AND  COMPUTER 
PROGRAMS 


EXISTING:  X 


PLANNED: 


STANDARDIZATION 

AREA:  MISC. 

PREPARING  ESD/PLEA ,  BEDFORD,  MA 

RICH  O'NEILL,  617-377-2703 
HQ  AFSC/SDX,  ANDREWS  AFB ,  WASHINGTON, 
D.  C. 

MAJ.  MILLER,  301-981-3316 

DOCUMENT  PURPOSE: 


This  standard  prescribes  the  requirements  for  the  con¬ 
duct  of  technical  reviews  and  audits  on  systems,  equipments, 
and  computer  software. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

The  appendices  in  MIL-STD-1521B  provide  the  guidance 
for  the  conduct  of  the  identified  technical  reviews  and 
audits  in  the  form  of  checklists.  These  checklists  contain 
the  purpose  of  the  particular  technical  review/audit  and  a 
list  of  items  to  be  reviewed.  The  items  to  be  reviewed  con¬ 
form  to  the  system  engineering  process  and  are  basically 
generic.  The  design  of  the  diagnostic  capability  is 
reviewed  and  evaluated  through  the  conduct  of  technical 
reviews  and  audits,  in  accordance  with  the  requirements  of 
the  standard. 
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GENERAL  RECOMMENDATIONS: 


Several  of  the  appendices  in  the  standard  should  con¬ 
tain  additional  diagnostic  capability-related  review  items 
to  ensure  that  the  diagnostic  capability  is  properly 
evaluated  and  measured.  Collation  of  several  diagnostic 
capability  review  items  should  be  accomplished  in  several 
appendices  in  order  to  provide  more  clarity  and  effective¬ 
ness  in  the  diagnostic  capability  review  process. 

SPECIFIC  MODIFICATIONS: 

The  following  additions/changes  to  the  identified 
review/audit  are  recommended. 

Appendix  A  —  Systems  Requirements  Review  (SRR) 

Page  19,  paragraph  10. 3h,  2nd  line:  Between  the  words 
"analysis,"  and  "armament",  insert  "diagnostic  capability 
analysis , " . 

Appendix  B  —  System  Design  Review  (SDR) 

Page  24,  paragraph  20.2.3d,  3rd  line:  Between  the  words 
"analysis,"  and  "maintainability",  insert  "diagnostic 
capability  analysis,". 

Page  25,  paragraph  20.3.1:  After  subparagraph  "g.",  insert 
a  new  subparagraph  "h.  Diagnostic  Capability"  and  reletter 
each  subsequent  subparagraph  alphabetically,  beginning  with 
the  letter  "i.". 

Page  26,  paragraph  20.3.2h:  Change  the  first  word, 

"Testability"  to  "Diagnostic  Capability". 

Page  28,  paragraph  20.3.11:  After  subparagraph  "a.",  insert 
a  new  subparagraph  "b.  Diagnostic  capability  requirements 
in  the  updated  system/segment  specification".  Reletter  the 
subsequent  subparagraphs  alphabet ically ,  beginning  with  the 
letter  "c.". 

Appendix  D  —  Preliminary  Design  Review  (PDR) 

Page  34,  paragraph  40.2.1m:  Between  "availability"  and 
"data",  insert  "/diagnostics  capability". 
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Page  37,  paragraph  40.2.3:  After  subparagraph  "f.",  insert 
a  new  subparagraph,  "g.  Review  support  equipment  diagnostic 
capability"  and  reletter  subsequent  subparagraphs  alphabeti¬ 
cally,  beginning  with  the  letter  "h.". 

Page  38,  paragraph  40.2.3.1,  2nd  sentence:  Change  the  sen¬ 
tence  to  read,  "Relate  this  to  the  development  of  Built-In 
Test  (BIT) /Built-In  Test  Equipment  (BITE),  Testability  at¬ 
tributes,  and  try  to  reduce  the  need  for  complex  support 
equipment. " 

Page  40,  subparagraph  40.6.7:  Delete  paragraph  40.6.7. 

Page  42:  Insert  a  new  paragraph  40.8,  as  follows: 

"40.8  Diagnostic  Capability 

40.8.1  Identify  the  quantitative  diagnostic 
capability  requirements  specified  in  the  hardware 
development  and  software  requirements  specifica¬ 
tions;  if  applicable,  compare  preliminary  pre¬ 
dictions  with  specified  requirements. 

40.8.2  Review  the  results  of  the  analysis  of  the 
inherent  (intrinsic)  testability  of  the  prelimi¬ 
nary  design. 

40.8.3  Identify  the  BIT/BITE,  status  monitoring, 
fault/detection,  fault/isolation  levels;  if  ap¬ 
plicable,  compare  preliminary  predictions  with 
specified  requirements. 

40.8.4  Review  the  sensitivity  of  diagnostic 
capability  quantification  parameters  (FD/FI 


levels) 

on : 

a . 

Maintainability  (MTTR) 

b. 

Reliability  (MTBF) 

c . 

Manpower  and  Personnel 

Requirements 

d. 

Training  Requirements 

e . 

Technical  Information 

Requ i rements 

f . 

Off-Equipment  Testing 

Requ i rements . 
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Renumber  existing  paragraph  "40.8  System  Safety"  to 
"40.9,"  and  add  one  digit  to  each  subsequent  existing 
paragraph  (e.  g.,  40.10  to  40. 11, etc.). 

Page  48,  paragraph  40.14.2:  After  the  first  sentence  add 
the  following  in  parentheses,  "(including  diagnostic 
capability-related  data  items.)" 

Appendix  E  —  Critical  Design  Review  (CDR) 

Page  54,  paragraph  50.2.1c:  Between  current  listings  "(8) 
and  (9),"  add  a  new  listing:  "(9)  Diagnostic  Capability". 
Renumber  each  subsequent  listings  by  increasing  it  by  one 
(e.  g.,  existing  9  to  10,  etc.). 

Page  61:  Insert  new  paragraph,  as  follows: 

"50.8  Diagnostic  Capability 

50.8.1  Review  the  most  recent  predictions  of 
diagnostic  capability  quantitative  values  and  com¬ 
pare  these  against  requirements  specified  in  the 
HWCI  Development  Speci f ication  and  Software  Re¬ 
quirements  specification. 

50.8.2  Review  compatibility  of  HWCI  and  CSCI 
diagnostic  capability  quantitative  values  with  ex¬ 
ternal  diagnostic  capability  detail  specifica¬ 
tions  . 


50.8.3  Identify  unique  diagnostic  capability  pro¬ 
cedures  required  for  the  configuration  item  during 
operational  use  and  evaluate  their  total  effect  on 
system  maintenance  concepts  and  support  require¬ 
ments. 


50.8.4  Review  detailed  maintainability  demonstra¬ 
tion  plan  for  inclusion  of  diagnostic  capability 
test  requirements." 

Renumber  existing  paragraph,  "50.8  System  Safety"  to 
"50.9",  and  renumber  each  subsequent  paragraph  in  numerical 
order  (e.  g.,  existing  50.9  to  50.10,  etc.). 

Page  64,  paragraph  50.14.3:  On  second  line,  insert 
"Diagnostic  Capability"  between  "Maintainability"  and 
"Data". 
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DOCUMENT  IMPROVEMENT  REPORT 


REQUIREMENT  NO.:  6 

REQUIREMENT  TITLE:  ASSURING  THE  DELIVERY  OF  AN  ADEQUATE 

DIAGNOSTIC  CAPABILITY 

DOCUMENT  NUMBER:  MIL-STD-471A 

TITLE:  MAINTAINABILITY  VERIFICATION/DEMONSTRA¬ 

TION/EVALUATION 

EXISTING:  X  PLANNED: 

STANDARDIZATION 

AREA:  MISCELLANEOUS 

PREPARING  RADC/RBE-2,  ROME,  NY 

ACT I VT T Y :  JOSEPH  CAROLI ,  315-330-4205 

DOCUMENT  PURPOSE: 

MIL-STD-4  7 1A  provides  procedures  and  test  methods  for 
verification,  demonstration,  and  evaluation  of  qualitative 
and  quantitative  maintainability  requirements.  It  also 
provides  for  qualitative  assessment  of  various  integrated 
logistic  support  factors  related  to  and  impacting  the 
achievement  of  maintainability  parameters  and  item  downtime, 
e.  g.,  technical  manuals,  personnel,  tools  and  test  equip¬ 
ment,  maintenance  concepts  and  provisioning. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

MIL-STD-471A  ,  Maintainability 

Verification/Demonstration/Evaluation  is  the  logical  re¬ 
quirements  document.  This  establishes  provisions  for  tes¬ 
tability  and  diagnostics  demonstration  related  to  fault 
detection  and  fault  isolation  capability  provided  by  the  in¬ 
tegrated  diagnostics  elements,  including  built-in  test,  test 
equipment,  technical  manuals,  and  skills. 


GENERAL  RECOMMENDATIONS: 

MIL-STD-471A  currently  addresses  demonstration  of 
"maintainability  tasks  and  procedures,"  which  by  definition 
includes  diagnostics  procedures.  However,  emphasis  is  not 
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given  to  diagnostics  (fault  detection  and  fault  isolation) 
parameters.  It  is  recommended  that  diagnostics  procedures 
to  be  demonstrated  be  emphasized  within  MIL-STD-471  and  that 
formal  linkages  to  MIL-STD-2165  be  provided. 

SPECIFIC  MODIFICATIONS: 


Paragraph  1.1: 

Change  "maintainability  requirements"  to  main¬ 
tainability  and  diagnostics  requirements". 


Change  "maintainability  parameters"  to  "maintainability 
and  diagnostics  parameters". 


Change  last  line  to:  "e. 
manuals  and  maintenance  aids, 
tools  and  test  equipment, 
provisioning . " 

Section  2: 


g.,  built-in  test,  technical 
personnel  and  skill  levels, 
maintenance  concepts  and 


Include  MIL-STD-2165  as  a  reference. 
Include  MIL-STD-721C  as  a  reference. 


Section  3: 


Include  Testability  and  Diagnostics  definitions. 
Section  4: 


Add  at  end  of  paragraph  4.1: 

"j.  A  testability  demonstration  shall  be  performed  in 
conjunction  with  the  maintainability  demonstration 
and  in  accordance  with  the  maintainability  test 
plan. 

The  testability  demonstration  is  designed  to: 

1.  Ascertain  if  operational  system  checks  can 
detect  the  presence  of  failures,  per  the 
development  specification. 

2.  Ascertain  if  system/equipment/module  BIT  can 
detect  and  isolate  failures,  per  the  develop¬ 
ment  specification. 
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3. 


Ascertain  if  each  UUT 
ATE (s)  . 


is  compatible  with  the 


It  shall  be  performed  in  accordance  with  the  re¬ 
quirements  of  8  December  1978  addenda  to  this  standard 
titled  "Maintainability  Verification/Demonstration/ 
Evaluation . " 


Appendix  B  Modifications: 


o  A  family  of  curves  for  various  confidence 
levels  is  required. 

o  The  procedures  identify  a  symbol  that  re¬ 
presents  sample  size.  Nowhere  in  the  pro¬ 
cedure  is  the  sample  size  defined.  This  is 
required  in  order  to  use  the  procedure. 

o  A  sequential  analysis  technique  must  be  in¬ 
cluded  or  the  insertion  of  multiple  faults 
must  be  addressed. 

o  Provide  formulae  to  compute  false  alarm  rate 
acceptance/rejection  criteria,  based  on 
confidence  levels." 
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DOCUMENT  IMPROVEMENT  SHEET 


REQUIREMENT  NO.: 

8 

REQUIREMENT  TITLE: 

STANDARDIZATION  OF  DEFINITIONS 

DOCUMENT  NUMBER: 

(1)  MIL-STD-721C 

TITLE: 

DEFINITIONS  OF  TERMS  FOR  RELIABILITY 
AND  MAINTAINABILITY 

DOCUMENT  NUMBER: 

(2)  MIL-STD-1309C 

TITLE: 

DEFINITION  OF  TERMS  FOR  TEST,  MEASURE¬ 
MENT,  AND  DIAGNOSTIC  EQUIPMENT 

EXISTING:  X 

PLANNED: 

STANDARDIZATION 

(1)  RELIABILITY 

AREA: 

(2)  ATTS 

PREPARING 

(1)  NAVAIR,  51122,  WASHINGTON,  D.  C. 

ACTIVITY: 

JOHN  COOK,  201-323-7458,  NAEC 
(2)  NAVSEA  55Z3 ,  WASHINGTON,  D.  C. 
JEAN  HARMON,  202-692-0160 

DOCUMENT  PURPOSE: 

There  are  two  standards  which  are  dedicated  to  defining 
diagnostic  and  testability  terms. 

The  first  is  MIL-STD-721C,  Definition  of  Terms  for 
Reliability  and  Maintainability.  This  document  is  dated  12 
June  1981.  It  defines  words  and  terms  most  commonly  used, 
which  are  associated  with  reliability  and  maintainability. 
It  is  under  the  reliability  standardi zation  area,  and  NAVAIR 
is  the  preparing  activity.  It  contains  a  definition  of  128 
terms,  many  of  which  are  inherently  testability  and  diagnos¬ 
tic  terms.  However,  the  terms  "diagnostics”  and 
"testability"  are  not  included.  This  standard  more  closely 
fits  the  definition  of  a  standard  because  most  of  the  terms 
used  are  mutually  exclusive  of  one  another. 

The  second  standard  is  MIL-STD-1309C,  which  was  issued 
18  November  1983.  The  title  of  this  standard  is  "Definition 
of  Terms  for  Test,  Measurement,  and  Diagnostic  Equipment." 
The  definitions  that  are  included  in  this  standard  are  ex- 
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tracted  from  a  number  of  other  standards,  handbooks, 
specifications,  and  other  documents.  Although  the  standard 
relates  to  TMDE,  many  of  the  terms  do  not  have  a  direct 
relationship  to  TMDE  (e.  g.,  testability,  reliability, 
redundancy).  The  document  has  681  terms  defined.  It  is 
more  a  dictionary  than  a  standard  because  there  is  no  at¬ 
tempt  to  make  the  definitions  mutually  exclusive. 

There  is  some  duplication  between  the  two  standards. 
Thirty-one  of  the  terms  appearing  in  MIL-STD-721  also  appear 
in  MIL-STD-1309.  However,  only  four  of  these  terms  have  the 
same  definitions  in  both  documents.  That  is  not  necessarily 
to  say  that  the  two  definitions  are  in  conflict,  just  that 
they  differ  somewhat. 

There  are  diagnostic  and  testability  terms  which  are 
missing  from  both  documents.  Integrated  Diagnostics  is  not 
defined,  and  terms  (e.  g.,  expert  systems,  knowledge  en¬ 
gineering)  are  not  included. 

A  major  problem  exists  in  defining  the  content  of  these 
two  standards.  On  one  hand,  MIL-STD-721  does  not  include 
all  of  the  terms  relative  to  reliability  and  main¬ 
tainability,  if  indeed  diagnostics  and  testability  are  a 
subset.  On  the  other  hand,  MIL-STD-1309C  covers  terms  which 
not  only  apply  to  TMDE,  but  apply  to  many  terms  outside  of 
its  realm.  Separating  the  two  is  very  difficult. 
Reliability  and  maintainability  overlap  with  testability  and 
diagnostics  and,  if  the  components  of  diagnostics  (i.  e., 
testing,  training,  and  technical  information)  are  also 
depicted,  the  overlap  becomes  more  confusing. 

Thus,  it  does  not  seem  that  the  separation,  based  on 
the  titles  of  documents,  appears  realistic.  Where  tes¬ 
tability  and  diagnostic  terms  should  appear  is  also  hazy. 

TESTABILITY/DIAGNOSTIC  IMPACT: 

Conflicting  terms  and  terms  not  defined  both  have  a 
significant  effect  on  understanding  between  the  government 
and  its  contractors.  Trying  to  define  every  reliability, 
maintainability,  and  diagnostic  term  appears  unrealistic  and 
could  cause  confusion  by  not  standardizing  on  single  terms 
in  lieu  of  overlapping  terms. 
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GENERAL  RECOMMENDATIONS: 


The  NS IA  Integrated  Diagnostics  Group  recommended  com¬ 
bining  the  two  standards  into  one.  Although  this  recommen¬ 
dation  may  have  merit  in  the  long  run,  it  might  prove  to  be 
difficult  and  time  consuming. 

Another  alternative  is  to  expand  MIL-STD-721  to  include 
preferred  definitions,  including  those  for  testability  and 
diagnostics,  utilizing  a  mutually  exclusive  concept  for 
these  definitions.  On  the  other  hand,  MIL-STD-1309  could  be 
maintained  as  a  dictionary  of  relative  terms,  much  the  same 
as  it  appears  now,  but  assuring  that  there  is  no  conflict  in 
terms  between  the  two  standards.' 

SPECIFIC  MODIFICATIONS: 


An  analysis  of  principal  testability/diagnostic  terms 
is  shown  in  Figures  4-1  through-  4-4.  The  terms  are  clas¬ 
sified  under  four  major  headings:  Availability  (Measures); 
Diagnosis;  Test;  and.  Diagnostic  Capability.  Each  of  the  71 
terms  depicted  on  these  diagrams  is  keyed  as  follows: 


1. 

2. 

3. 


The  term  is 

The  term  is 

The  term  is 
a  principal 


presently  in  MIL-STD-721C. 

presently  in  MIL-STD-1309C. 

not  defined  in  either  standard, 
term. 


but  is 


NOt  OF  PRINCIPAL 
TESTABILITY/DIAGNOSTIC  TERMS 


MIL-STD-721C 

15 

MIL-STD  1309C 

43 

NOT  DEFINED 

22 

30  * 

*  Nine  of  the  terms  appear  in  both  standards. 
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It  is  recommended  that  54  terms  (34  from  MIL-STD-1309 
and  22  more  not  defined)  be  included  in  MIL-STD-721C  and 
that  this  standard  be  the  primary  standard  referenced  in 
contractual  documents  relating  to  these  types  of  defini¬ 
tions  . 
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V.  SUMMARY. 


This  report  addresses  the  necessary  modifications  to  24 
existing  standards  and  handbooks  required  to  assure  the  ac¬ 
quisition  of  an  adequate  weapon  system  diagnostic 
capability.  The  24  standards/handbooks  that  have  been 
recommended  for  testability/diagnostics-related  modifica¬ 
tions  are  shown  in  Table  5-1,  Testability/Diagnostics- 
Related  MIL-STDs  Hierarchical  Relationships.  The  table  has 
been  formulated  and  included  to  provide  the  following: 

o  The  relationships  of  the  standards  to  each  other 

o  The  depiction  of  the  testability/diagnostics-related 
standards  hierarchy,  which  includes  generic  stan¬ 
dards,  generic  electronic  standards,  and 
product/discipline-oriented  standards 

o  The  relational  impact  that  the  standards  recommended 
for  major  modification  have  on  the  hierarchy. 

The  modifications  recommended  for  all  the  standards  in 
Table  5-1  will  provide  a  cohesive  diagnostic  design  process 
that  is  implementable .  The  utilization  of  the  standards 
during  the  integrated  diagnostics  process  is  shown  in  Appen¬ 
dix  B. 


This  report  provides  a  firm  foundation  for  the  proper 
implementation  of  both  testability  and  diagnostics  in  the 
design  and  acquisition  of  weapon  systems. 
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table  5-1.  TESTABILITY /DIAGNOSTICS  RELATED  MIL-STDS  HIERARCHICAL  RELATIONSHIPS 
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APPENDIX  A 


LISTING  OF  EXISTING  AND  PLANNED  MILITARY  STANDARDS,  HAND¬ 
BOOKS  AND  GUIDES  APPLICABLE  TO  TESTABILITY  AND  DIAGNOSTICS 


Table  A-l  is  a  listing  of  existing  and  planned  docu¬ 
ments  which  conceivably  relate  to  testability  and  diagnos¬ 
tics  and  which  were  incorporated  in  this  analysis.  As  can 
be  seen,  these  documents  cover  a  wide  variety  of  standard¬ 
ization  areas,  indicative  of  the  testability  and  integrated 
diagnostics  process.  The  sixteen  (16)  documents  required  as 
a  minimum  in  the  RADC  Statement  of  Work  for  this  contract 
are  also  identified. 

The  proposed  disposition  of  each  of  the  documents  is 
also  indicated.  These  categories  are  as  follows: 

o  Major  Revision  --  Significant  changes  are  required 

o  Minor  Revision  —  Changes  include  word  and  para¬ 
graph  insertions  and  changes,  without  significantly 
changing  the  mission  of  the  document 

o  No  Action  Required  —  Document  is  satisfactory  in 
its  present  form 

o  Recommend  Canceling  --  Cancellation  or  nonuse  is 
recommended 

o  Guidance  Required  --  No  change  required.  However, 
material  in  these  documents  should  be  incorporated 
in  guidance  for  implementing  the  integrated  diagnos¬ 
tics  process. 
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TABLE  A- 1 .  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS 


TABLE  A-l.  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS  (Continued) 


TABLE  A— 1 .  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS  (Continued) 

T  l~  DISPOSITION 
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Aircraft/Store  Electrical  Inter-  GDRQ 


i-STD-  Software  Quality  Evaluation 


TABLE  A— 1 .  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS  (Continued) 
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TABLE  A-l.  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS  (Continued) 


Technical  Manual  Writing  Handbook 


TABLE  A-l .  LISTING  OF  EXISTING  AND  PLANNED  DOCUMENTS  (Continued) 


f 
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Guide  HFAC 


APPENDIX  B 


DETAILED  APPROACH 

1.0  SCOPE. 

This  appendix  contains  the  top-down  approach  used  for 
determining  documentation  requirements. 

1.1  Purpose. 

The  purpose  of  this  appendix  is  to  outline  the  approach 
and  methodology  used  to  show  relevance  of  the  MIL-STDs, 
MIL-HDBKs,  and  MIL-SPECs  selected  for  review  in  this  report 
to  the  diagnostic  activities  of  the  integrated  diagnostic 
process . 

2.0  APPROACH. 

A  top-down  approach  was  used  for  determining  documenta¬ 
tion  requirements  for  Phases  I  and  II  and  for  developing 
Phase  III  modifications. 

2.1  Document  Relationship  to  Weapon  System  Life  Cycle 

Phases. 

During  Phase  I,  the  format  used  for  determining  these 
relationships  is  shown  in  Figure  B-l,  MIL-STD/HDBK  Relation 
to  Weapon  System  Life  Cycle.  The  major  programmatic  stan¬ 
dards,  which  are  task-oriented,  are  tied  to  the  weapon  sys¬ 
tem  life  cycle  and  each  task  delineated  in  relation  to  this 
life  cycle. 

The  diagnostic  and  testability  activities  are  then 
depicted  in  relation  to  the  weapon  system  life  cycle  and  to 
the  tasks  described  in  the  programmatic  standards.  These 
diagnostics  and  testability  activities  were  established 
using  the  Integrated  Computer-Aided  Manufacturing  Definition 
(IDEF  0)  process,  which  grew  out  of  the  Integrated 
Computer-Aided  Manufacturing  System  (ICAM).  This  process, 
although  developed  for  ICAM,  can  be  applied  toward  the  plan¬ 
ning  of  most  any  program.  As  shown  in  Figure  B-2,  for  each 
diagnostic/testability  activity,  there  are  inputs  and  out¬ 
puts.  There  are  controls  over  this  activity,  such  as 
military  standards.  There  are  mechanisms  which  can  aid  in 
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Figure  B-l.  MIL-STD/HDBK  Relation  to  Weapon  System  Life  Cycle 


CONTROLS 

OTHER  CONTROLS  (e.  g.,  POLICY,  BUDGET) 


performing  this  activity,  such  as  military  handbooks.  In 
this  manner,  IDEF  0  was  used  to  determine  the  standard  and 
handbook  requirements.  Note  that  programmatic,  as  well  as 
process  or  product  type,  documents  are  addressed. 

This  same  format  was  used  during  Phase  II  as  a  basis 
for  comparison  of  existing  and  planned  standards  and  hand¬ 
books,  with  the  requirements  established  under  Phase  I  of 
the  program.  This  entire  process  (the 
diagnostic/testability  activities  and  the  applicable  docu¬ 
ments  for  these  activities)  is  contained  in  the  Integrated 
Diagnostics  Process  Roadmap  included  in  Section  2.4  of  this 
appendix . 

2.2  Integrated  Diagnostics  Process. 

The  Integrated  Diagnostics  Process  Roadmap  contains  the 
system  engineering  activities  that  have  a  relationship  to 
the  implementation  of  diagnostics  into  weapon  systems,  prime 
system,  subsystems,  equipment,  and  their  associated  support 
systems.  Therefore,  the  process  described  in  this  appendix, 
when  used  concurrently  with  an  emerging  system,  subsystem, 
or  equipment,  defines  the  requirements  for  specifying  diag¬ 
nostics;  allocating  diagnostics  to  system,  subsystem,  and 
unit  levels;  identifying  testability/diagnostic  tasks; 
designing  the  diagnostic  capability;  conducting  design 
reviews  and  audits;  assuring  the  delivery  of  an  adequate 
diagnostic  capability;  and  collecting  and  analyzing  data  on 
the  performance  of  the  diagnostic  capability. 

2.2.1  Roadmap  Purpose. 

The  purpose  of  the  roadmap  is  to  show  the  relevance  of 
the  MIL-STDs ,  MIL-HDBKs ,  and  MIL-SPECs  selected  for  review 
in  this  report  to  the  diagnostic  activities  contained  in  the 
roadmap.  An  analysis  of  the  match-up  between  the  documents 
and  the  diagnostic  activities  will  identify  "gaps"  and 
"shortfalls"  in  the  documents  relative  to  the  implementation 
of  the  Integrated  Diagnostics  process. 
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2.2.2  Analysis  Methodology 


The  determination  of  document  gaps  and  shortfalls  is  a 

two-step  process. 

STEP  (1)  --  Analyze  the  Integrated  Diagnostics 

Process  Roadmap  Activities  for  the  ab¬ 
sence  of  MIL-STDs  and/or  MIL-HDBKs  . 
Determine  for  a  particular  activity  the 
need  for  MIL-STD  and/or  MIL-HDBK. 

STEP  (2)  --  For  each  activity,  the  sufficiency  of 

the  requirements  contained  in  the  MIL-STD 
and  MIL-HDBK  for  that  particular  activity 
must  be  evaluated.  This  has  been  ac¬ 
complished  through  the  use  of  Document 
Improvement  Report  sheets  for  each 
selected  document,  which  are  contained  in 
Appendix  D. 

2. 2. 2.1  Analysis  Results. 

The  results  of  the  analysis  are  as  follows: 

CONCEPTUAL  PHASE: 

Activities  A123  --  One  programmatic  standard  is  needed 

to  A1235  which  addresses  all  the  testa¬ 

bility/diagnostic  elements. 

Activity  A1235  --  Guidance  for  determining  the  T/D 

impact  on  producibil ity  and  manu¬ 
facturing  is  needed  in  a  standard 
or  handbook. 

Activity  A1242  --  MIL-STD-1521B  should  be  revised  to 

address  diagnostics  during  System 
Requirements  Review  (SRR) . 

DEMONSTRATION  &  VALIDATION  PHASE: 

Activities  A134  --  One  programmatic  standard  is  needed 

to  A1346  which  addresses  all  the  test¬ 

ability/diagnostic  elements. 


Activity  A1342  —  Guidance  for  incorporating  diagnos- 

A1346  tic  parametric  values  into  systems, 

subsystems,  and  support  systems 
should  be  provided  in  standards 
and/or  handbooks. 

Activity  A1344  —  The  impact  of  new  technologies  and 

new  system  architectures  on  the 
weapon  system's  entire  diagnostic 
capability  must  be  addressed. 

Activity  A13444  —  T/D  Segments  for  Logistic  Support 

Modeling  --  Guidance  should  be 
included  in  a  handbook. 

Activity  A1362  —  T/D  Risk  Areas  Assessment  and  Tech¬ 

nology  Assessment  —  guidance 
for  performing  T/D  risk  and  tech¬ 
nology  assessment  is  needed  in  a 
standard  and/or  handbook. 

Activity  A1352  --  MIL-STD-1521B  should  be  revised  to 

include  diagnostic  review  during 
System  Design  Review  (SDR) . 

FULL-SCALE  ENGINEERING  DEVELOPMENT  PHASE: 

Activity  A14311  --  One  programmatic  standard  is  needed 

to  address  all  the  testability/ 
diagnostic  elements. 

Activity  A14312  —  Guidance  is  required  to  conduct 

diagnostic  element  tradeoffs  for 
optimization  of  diagnostic  cap¬ 
ability  and  weapon  system  design. 

Activity  A143113  —  Diagnostic  Parametric  Values  impact 

on  Technical  Information.  Guidance 
is  required  in  standards  and  hand¬ 
books  . 

Activity  A143114  --  Diagnostic  Parametric  Values  impact 

on  Manpower  and  Trainin'’. 

Guidance  is  required  in  standards 
and  handbooks. 
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Activity  A143115 


Diagnostic  Design  Criteria  Guidance 
is  needed  to  determine  the  impact 
of  on-line  diagnostic  decisions  on 
off-line  testing  requirements. 

Activity  A1432  —  Diagnostic  Data  Collection  --  All 

the  programmatic  standards  mention 
data  collection  but  none  address 
the  specific  data  collection  re¬ 
quirements  for  diagnostics. 

Guidance  material  on  specific  data 
required  must  be  prepared. 

Activity  A1433  --  MIL-STD-1521B  must  be  revised  to 

identify  d iag nost i c-spec i f ic 
review  items. 

Activity  A1435  —  Diagnostics  Design  Impact  on  Prime 

System  Mission  Availability 
Standards  listed  address  mission 
availability  without  providing  any 
details.  Guidance  to  determine 
diagnostics  impact  on  mission 
availability  is  needed  in  a  hand¬ 
book  . 

Activity  A1437  —  MIL-STD-1521B  must  be  revised  to 

identify  diagnostic-specific 
review  items. 

Activity  A148  --  Maintenance  Aids  Requirements 

Guidance  for  addressing  diagnostic 
parametric  value  selection  on 
maintenance  aid  requirements  should 
be  included  in  a  standard  or  hand¬ 
book  . 

PRODUCTION  PHASE: 

Activity  A25  --  Diagnostic  Performance  Assessment 

Activity  A251  —  Diagnostic  Data  Collection  and  Mat¬ 

uration  --  More  detailed  guidance 
for  diagnostic  data  collection  and 
performance  assessn nnt  during  pro¬ 
duction  is  needed. 
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DEPLOYMENT  PHASE: 

Activity  A342  —  Diagnostic  Data  Base  Requirements 

The  requirements  must  be 
defined  and  included  in  a  handbook. 

2.3  Conclusions. 

From  the  analysis  of  the  Integrated  Diagnostics  Process 
Roadmap  activities,  the  following  conclusions  were  formu¬ 
lated  . 

1.  The  programmatic  standards,  while  addressing  all  of  the 
diagnostic  issues  fragment  the  integrated  diagnostic  process 
so  that  the  process  for  developing  the  diagnostic  capability 
is  not  readily  apparent  and  difficult  to  implement. 

2.  The  activities  identified  above  which  indicate  a  gap  or 
a  shortfall  in  process/product  oriented  standards  and  hand¬ 
books  could  be  corrected  through  the  implementation  of 
planned  handbooks  and  the  modification/revision  of  existing 
process/product  oriented  standards  and  handbooks. 

3.  The  Integrated  Diagnostics  Process  Roadmap  should  be 
used  as  the  foundation  for  defining  and  providing  guidance 
for  the  implementation  of  the  Integrated  Diagnostics 
Process . 

2.4  Integrated  Diagnostics  Process  Roadmap. 

Figures  B-3  through  B-8  are  the  Roadmap  Flow  Charts. 
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VITIES  THAT  REQUIRE  ASSOCIATED  MIL-STDs/MIL-HDBKs 


HIL-STD-470A 

7202.2 . 1 
HIL-STI-499A 
Pira.K  .2.9 
MH.-STI-13M-1A 

7205.2.2 
M2L-STC-216S 
7201.2. J. 7204.4. ^ 


MIL-STD-490A 
ADpandlx  A 


Kl2.-STS-152ia 


DEMONSTRATION  &  VALIDATION  PHASE  INTEGRATED  DIAGNOSTIC  PROC 


KIL-STD-499A  Y 

Para.i.L.lO.lJU 
Aizi:  j 


fKlL-STX>-454J 

L  **»*--  «  « 


HXL-*Tt>-470 

T206.2 

MIL-STD-49W 

Mtt.  10.2.1.10.2.2 

KIL-STD-7I5I 

T201.2 

MIL-STO-13IO-U 
T202 .2.4.7702.2, 
T202.2.T204.2, 
T301.2.T401.2 
WW7&-2145 
T201.2.T202.2 


A23< 


ostine  ?/d  input 

TO  SYSTEMS  ENG  A. 
SECTION  Of  PKP 


KIL-STO-4  70A  *1 

T301.2 

KIL-STD-499A 
Para.10.1.3 
MIL -STD- 13  88—  LA 
TSOI. 2 
MIL-STD-2165 
7301.2.3 


AID  1 3 


DEflNE  INPUTS  TO 
TIE  SECTION  Of  PKP 


KIL-S7D-499A 
Para. 5.1 
MIL-STO-Ull- 
7101. 2.1 


V. 


MIL- STD-  2155 
KIL-STD-4  71A 


A1313 


i 


OtriNE  ?/S  INPUTS 
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SECTION  I 


INTRODUCTION 


The  Rome  Air  Development  Center  (RADC) , 
the  Office  of  the  Secretary  of  Defense, 
Testability/Diagnostics  Encyclopedia  Program, 
of  this  program  is  aimed  at  reviewing  existing 
in  the  fields  of  engineering,  reliability, 
logistics,  technical  information,  software, 
respect  to  essential  testabi 1 i ty/d i ag nost i c 
product  from  the  first  portion  of  this  program 
revisions  to  these  Standards. 


with  support  from 
is  sponsoring  a 
The  first  portion 
Military  Standards 
maintainability, 
and  training  with 
coverage.  The 
will  be  proposed 


The  analysis  of  these  Standards  has  been  completed.  Before 
embarking  on  further  definition  of  required  revisions  and  iden¬ 
tification  of  additional  needs,  RADC  encouraged  coordination  with 
a  wide  variety  of  DoD  and  industry  personnel  engaged  in  the 
utilization  of  these  Standards.  An  Interim  Report  was  prepared 
and  distributed  which  contained  the  results  of  this  analysis. 


At  the  request  of  the  Rome  Air  Development  Center,  the  Na¬ 
tional  Security  Industrial  Association's  (NSIA)  Integrated  Diag¬ 
nostics  Group  reviewed  this  Interim  Report  and  developed  recom¬ 
mendations  for  improvement  in  the  military  Standards  which  con¬ 
trol  the  acquisition  of  testability  and  the  weapon  system's  diag¬ 
nostic  capability.  The  results  of  this  analysis  are  contained  in 
this  report. 

The  NSIA  Integrated  Diagnostics  Group  met  twice  to  conduct 
this  review  and  analysis.  The  first  meeting  was  in  Pinehurst, 
North  Carolina,  on  15-16  July  1987.  The  second  meeting  was  on  27 
August  1987  in  Anaheim,  California.  Appendix  A  is  a  list  of  par¬ 
ticipants  at  both  the  Pinehurst  and  Anaheim  meetings. 

A  workshop  environment  was  established  at  both  meetings, 
with  the  specific  purposes: 

o  To  help  focus  attention  of  industry  on  needed  Standards, 
which  are  essential  to  enable  the  implementation  of 
testability  and  diagnostics  into  the  design  process. 

o  To  examine  current  military  Standards  effort  for  effect- 
ness  in  the  implementation  of  both  testability  and  diag¬ 
nostics. 

o  To  identify  and  define  specific  Standards,  modifications, 
and  developments  which  are  most  needed  and  could  be 
accomplished  in  the  near  future. 


C-3 


o  To  identify  possible  methods  to  accelerate  development 
and  implementation  of  these  Standards. 

Section  II  of  this  report  contains  a  summary  of  the  conclu¬ 
sions  reached  during  these  meetings.  Section  III  contains  the 
findings  and  recommendations  for  each  of  the  military  Standards 
which  the  group  reviewed  in  relation  to  eight  requirements  which 
were  established  during  the  early  phase  of  the  RADC  Program. 
Section  IV  is  a  summary  of  the  report,  with  required  actions 
identified. 
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SECTION  II 


CONCLUSIONS 


Although  the  discussion  and  recommendations  for  each  of  the 
applicable  military  Standards  are  contained  in  Section  III,  dis¬ 
cussion  at  these  review  meetings  centered  around  a  number  of 
positive  issues  relating  to  these  Standards.  These  discussions 
resulted  in  a  number  of  conclusions  which  have  wide  implications 
in  the  content  and  use  of  these  Standards.  These  conclusions  can 
be  summarized  as  follows. 

o  Strengthen  the  development  of  a  more  effective  and 

efficient  diagnostic  capability  by  promoting  diagnostics/ 
testability  as  in  integral  part  of  the  system  engineering 
process. 

With  the  advent  of  the  term  "Integrated  Diagnostics,"  there 
is  a  tendency  to  invent  a  new  "ility"  to  control  the  development 
of  the  diagnostic  capability.  However,  Integrated  Diagnostics  is 
a  process  for  the  integration  of  all  the  diagnostic  elements  that 
contribute  to  diagnostics  (e.  g.,  f au 1 t - t o 1 e r a n t  design, 
testability,  testing,  technical  information,  and  personnel  and 
training).  Diagnostics  is  mission  and  performance  oriented  and 
should  be  measured  as  such.  The  concept  of  diagnostic  growth 
should  be  employed  in  analyzing  an  demonstrating  the  performance 
of  the  diagnostic  capability.  Thus  MIL-STD-499,  which  deals  with 
the  system  engineering  process,  is  a  key  Standard,  which  requires 
modification  to  implement  Integrated  Diagnostics  and  testability. 


o  Integration  of  the  "ilities"  is  required  to  provide  a 
cohesive  diagnostic  design  and  evaluation  process. 

The  present  diagnostic/testability  design  process  is  con¬ 
trolled  by  a  number  of  programmatic  and  process  Standards.  These 
Standards  deal  with  reliability,  maintainability,  human 
engineering,  safety,  testability,  training,  logistic  support 
analysis,  test  requirements  documents,  test  program  sets,  etc. 
There  was  a  strong  sentiment  toward  minimizing  the  number  of 
Standards  with  duplicate  or  overlapping  requirements  that  are  ap¬ 
plied  to  a  weapon  system  program.  These  Standards  require 
program  plans,  tradeoff  analyses,  and  demonstrations,  among  other 
actions — all  of  which  have  a  direct  effect  on  the  design  of  the 
diagnostic  capability.  In  the  far-term,  combinations  of  these 
Standards  should  be  promoted  to  provide  a  more  cohesive  diagnos¬ 
tic  and  support  capability.  In  the  short-term,  methodologies 
should  be  developed,  such  as  tailoring  through  the  use  of  DIDs, 
to  lessen  the  impact  of  fractionated  "ilities." 
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o  Resist  the  temptation  to  develop  additional  Standards. 
Modification  of  present  Standards  to  address  diagnostics 
and  testability  appears  feasible.  Put  requirement 
provisions  in  Standards  and  "how  to"  information  in 
guides  and  handbooks. 

The  group  found  no  requirement  for  new  Standards,  except  in 
the  area  of  maintenance  aiding  and  maintenance  training  for 
electronic  delivery  of  this  type  of  information.  The  group 
rejected  the  need  for  a  Standard  for  embedded  diagnostics  design. 
The  group  identified  a  number  of  Standards  which  presently  con¬ 
tain  specific  guidance  or  tools  used  in  the  diagnostic  design 
process.  Examples  of  this  were  MIL-STD-454J,  which  deals  with 
design  criteria  for  electronic  systems,  and  MIL-STD-756,  which 
provides  detailed  methodologies  for  predicting  reliability.  The 
trend  should  be  to  minimize  placing  this  type  information  in 
Military  Standards  in  order  to  allow  contractors  as  much  freedom 
as  possible  to  do  their  job.  In  fact,  the  group  rejected  the 
need  for  a  testability  or  a  diagnostic  analysis  Standard,  but 
recommended  that  such  tools  be  incorporated  in  handbooks. 

o  Military  Standards  should  reflect  new  weapon  system 

architecture,  such  as  the  development  of  Common  Advanced 
Integrated  Avionics. 

New  weapon  system  architecture  which  incorporates  dynamic 
reconfigurability,  complex  redundancy,  and  graceful  degradation, 
requires  a  rethinking  of  the  Standards  used  in  the  diagnostic 
design  process.  Questions  arose  on  whether  the  present  failure 
modes  and  effects  criticality  analyses  were  able  to  effectively 
address  this  new  technology.  Other  new  technologies,  such  as  the 
electronic  delivery  of  technical  information;  the  employment  of 
VHSIC  circuitry;  expert  systems  technology;  and  the  integration 
of  the  diagnostic  design  process,  with  CAE/CAD/CALS;  are  all 
issues  which  will  drive  the  requirements  for  Military  Standards. 

o  There  is  a  need  to  strengthen  the  requirements  portions 
of  Standards  and  lessen  the  paper  deliverables. 

As  indicated  above,  there  appears  to  be  duplication  among 
the  paper  deliverables  invoked  by  a  number  of  Standards  and  as¬ 
sociated  DIDs.  A  significant  recommendation  was  made  for  MIL- 
STD-2165,  which  dealt  with  the  strengthening  of  the  requirements 
portion  of  this  Standard  to  not  only  address  testability,  but  the 
entire  scope  of  diagnostics,  and  on  the  other  hand,  reduce  the 
program  monitoring,  control,  and  demonstration  requirements  by 
placing  the  requirement  for  these  functions  in  other  existing 
Standards. 
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o  Maintainability  demonstration  techniques  need  to  be 
updated. 

In  concert  with  the  abovementioned  diagnostic  growth  concept, 
the  maturation  of  the  diagnostic  capability  must  be  planned, 
funded,  and  implemented.  Demonstrations  should  be  combined, 
where  feasible.  Data  collected  throughout  the  design  and 
demonstration  process  should  be  collected  and  become  the  baseline 
for  measuring  the  diagnostic  growth  of  the  system. 

o  Diagnostic  field  performance  data  must  be  defined, 
collected,  and  analyzed. 

Each  of  the  existing  programmatic  Standards  requires  that  a 
data  collection  and  analysis  system  be  established  for  reporting, 
tracking,  and  measurement  of  the  system’s  performance  in  relation 
to  each  of  the  "ilities."  Again,  the  group  did  not  recommend  a 
separate  diagnostic  data  base  but,  rather,  the  integration  with 
other  existing  or  planned  data  bases.  The  use  of  algorithms  that 
are  contained  in  RADC-developed  handbooks  to  predict  fault 
detection/fault  isolation  and  false  alarm  levels  identifies  the 
data  that  is  needed  as  inputs  to  the  algorithms.  When  conducting 
prediction,  empirical  data  is  used  as  inputs.  The  collection  of 
actual  data  and  its  use  an  inputs  to  the  algorithms  provides  the 
actual  fault  detection  and  fault  isolation  levels.  Field  data 
collection  presents  a  more  difficult  problem.  The  Standards 
should  provide  for  requirements  for  the  contractor  to  define 
field  diagnostic  performance  data  requirements  and  provide  for 
the  compatibility  with  the  data  collection  system  used  during 
design  with  that  utilized  in  the  field. 


SECTION  III 


FINDINGS/RECOMMENDATIONS 


Requirement  #1;  Establishing  Diagnostic  Requirements  and 

Allocating  These  Requirements  for  System, 
Subsystem,  and  Unit  Levels. 

MIL-STD-756  --  Reliability  Modeling  and  Prediction. 

Presently,  MIL-STD-756B  addresses  both  structured 
reliability  analysis  and  mission  reliability  analysis.  This 
Standard  addresses  conventional  architectures,  as  opposed  to 
newer  architectures,  which  have  complex  redundancy,  dynamic 
reconfigurability,  and  configurations  allowing  graceful 
degradation.  it  is  recommended  that  this  Standard  not  be  revised 
at  the  present  time  to  address  these  newer  architectures,  since 
the  technology  is  not  as  stable.  This  type  of  prediction  and 
modeling  can  be  adapted  to  satisfy  diagnostic  needs  and  should  be 
addressed  in  a  Military  Handbook  or  similar  document,  rather  than 
in  a  Military  Standard. 

MIL-STD-1591  —  Analysis/Synthesis  of  On-Aircraft  Fault 

Diagnosis,  Subsystems. 

This  Standard  establishes  uniform  criteria  for  conducting 
trade  studies  to  determine  the  optimal  design  for  on-aircraft 
fault  diagnosis/isolation  systems.  There  is  a  need  to  expand 
this  methodology  to  address  all  types  of  equipment  and  all  types 
of  diagnostic  elements  within  these  equipments  and  systems. 
However,  it  is  recommended  that  the  requirement  for  such  analyses 
be  included  in  the  revised  MIL-STD-2165  and  that  the  tools  and 
techniques  be  included  in  a  Military  Handbook  or  guide. 

Requirement  #?:  Describing  Various  Testability/Diagnostic  Tasks 

Which  Must  be  Undertaken  During  Each  Phase  of 
Weapon  System  Acquisition. 

MIL-STD-2165  —  Testability  Program  for  Electronic  Systems  and 

Equipments . 

MIL-STD-2165  was  initially  prepared  to  influence  the  design 
and  integration  of  testability  into  the  acquisition  process  for 
electronic  systems  and  equipments.  As  such,  it  is  very  close  to 
being  a  standard  which  implements  the  concept  of  Integrated 
Diagnostics.  A  very  significant  modification  was  proposed  by  the 
group.  The  intent  of  this  proposal  was  to  strengthen  the  design 
and  integration  portion  of  this  Standard,  while  lessening  the 
requirements  for  program  monitoring  and  control  and  test  and 
evaluation  by  utilizing  other  existing  Standards  to  perform  these 
functions.  Specific  recommendations  follow. 
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Task  101  —  Testability  Program  Planning. 

Task  101  calls  for  the  development  of  a  Testability  Program 
Plan  or  a  testability  plan  which  is  integrated  into  the  SEMP  or 
other  management  plans.  Integrated  Diagnostics  is  part  of  the 
system  engineering  process  and  thus  should  be  stressed  in  the 
System  Engineering  Management  Plan  (SEMP) .  The  SEMP  requirement 
is  usually  contained  in  the  weapon  system  Statement  of  Work,  un¬ 
der  the  Systems  Engineering  Process,  as  defined  by  MIL-STD-499. 
The  diagnostic  requirements  can  be  addressed  by  the  other  "ility" 
plans.  The  ultimate  recommendation  is  to  place  Integrated  Diag¬ 
nostics  into  the  SEMP,  along  with  the  other  "ility"  plans.  This 
would  require  a  major  modification  to  a  number  of  Standards,  as 
well  as  to  their  associated  DIDs.  The  near-term  approach  would 
be  to  emphasize  integration  of  diagnostics  in  SEMP,  while  tailor¬ 
ing  the  diagnostic  inputs  to  other  "ility"  program  plans  through 
the  use  of  DIDs. 

Task  102  —  Testability  Reviews. 

The  recommendation  is  to  put  adequate  testability/diagnostic 
review  criteria  in  MIL-STD-1521 ,  Technical  Reviews  and  Audits  for 
Systems,  Equipment,  and  Computer  Programs. 

Task  103  —  Testability  Data  Collection  and  Analysis  Planning. 

Each  of  the  programmatic  Standards  addresses  the  need  for 
data  collection  and  analysis.  MIL-STD-2165  addresses  the  need 
for  planning  for  this  function.  Therefore,  other  programmatic 
Standards,  such  as  MIL-STDs  785  and  470,  can  be  tailored  to 
address  this  requirement  by  integrating  the  data  needs  with  those 
already  required  for  reliability  and  maintainability. 

Task  301  —  Testability  Inputs  to  Maintainability 

Demonstration. 

This  task  already  puts  the  burden  of  demonstrating  tes¬ 
tability  and  diagnostics  on  MIL-STD-471. 

Task  200  Series 

This  series  addresses  Testability  Requirements,  Testability 
Preliminary  Design  and  Analysis,  and  Testability  Detailed  Design 
Analysis.  While  this  Standard  addresses  testability  in  a  broad 
sense,  it  must  be  expanded  to  fully  address  all  diagnostic  ele¬ 
ments  which  constitute  the  diagnostic  capability  (i.  e.  ,  testing, 
technical  information,  and  personnel  and  training)  .  In  addition, 
a  number  of  other  diagnostic  and  Integrated  Diagnostic  concepts 
must  be  addressed.  Among  these  are:  additional  emphasis  on 

translating  mission  requirements  into  FD/FI  requirements; 
tradeoff  analyses;  the  diagnostic  growth  concept;  diagnostic 
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allocations;  and  maturation  methodologies.  A  new  200  Series  of 
tasks  should  be  developed,  dealing  with  such  subjects  as 
allocation,  testability,  performance  monitoring, 
reconfigurability,  BIT,  and  nonembedded  diagnostics,  and  the 
title  of  the  document  should  be  changed  to  reflect  all  diagnostic 
elements . 

In  addition,  the  scope  of  this  Standard  should  not  be 
limited  to  electronics,  but  should  address  the  entire  weapon 
system's  diagnostic  capability. 

MIL-STD-470A  —  Maintainability  Program  Requirements. 

The  relationship  between  this  Standard  and  other  program¬ 
matic  Standards  is  not  clear  (e.  g.,  MIL-STD-1388-1A  and  MIL-STD- 
2165).  This  needs  to  be  clarified. 

MIL-STD-499A  —  Engineering  Management. 

This  Standard  provides  the  basis  for  Integrated  Diagnostics. 
However,  diagnostics  is  neither  identified  nor  addressed.  The 
Standard  should  be  modified  to  assure  that  Integrated  Diagnostics 
is  made  an  integral  part  of  the  system  engineering  process. 

MIL-STD-1629A  —  Procedures  for  Performing  a  Failure  Mode, 

Effects,  and  Criticality  Analysis. 

The  outputs  from  this  Standard  are  applied  to  a  number  of 
other  applications  and  Standards  (e.  g.,  testing,  safety, 

maintainability).  In  addition,  modern  dynamic  reconf igurable 
systems  require  a  re-look  at  FMECA  techniques.  This  Standard 
must  be  analyzed  completely  to  see  how  it  can  be  better  utilized 
in  providing  a  cohesive  diagnostic  design  process. 

DOD-STD-2167  —  Defense  Systems  Software  Development. 

This  Standard  deals  with  both  operational  and  diagnostic 
software.  It  is  recommended  that  only  diagnostic  software  and 
its  interface  with  operational  software  be  addressed  by  the  diag¬ 
nostic  community. 

MIL-STD-1379C  —  Contract  Training  Program. 

This  Standard  emphasizes  normal  methods  of  training,  such  as 
the  utilization  of  technical  manuals  for  instructional  material. 
Although  one  section  deals  with  on-the-job  training  handbooks, 
the  Standard  does  not  address  on-the-job  training  in  relation  to 
electronic  delivery  devices.  This  new  methodology  must  permeate 
throughout  this  Standard.  In  addition,  the  electronic  delivery 
device  can  not  only  deliver  on-the-job  training  information,  but 
also  can  deliver  technical  information  which  is  normally 
contained  in  a  technical  order  or  manual.  The  interface  between 
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these  two  types  of  information  must  be  described  and  methods  for 
generating  this  information  must  be  made  compatible. 

Requirement  #3:  Designing  the  Diagnostic  Capability. 

MIL-STD-454J  —  Standard  General  Requirements  for  Electronic 

Equipment . 

This  Standard  contains  general  requirements  for  electronic 
equipment,  which  are  oriented  toward  hardware.  Reliability, 
maintainability,  and  human  engineering  are  included,  with  the 
proviso  that  they  must  not  be  referenced  in  contractual 
documents.  It  is  recommended  that  Integrated  Diagnostics  not  be 
added  to  this  Standard  and,  if  any  of  the  diagnostic  elements  are 
included,  they  should  be  hardware  related.  The  consensus  was 
that  MI  L-STD-4  54  is  a  "handshake"  type  document  that  only 
benefits  the  Specification  preparer  and  not  the  user.  In 
addition,  it  is  recommended  that  Requirement  #  32  in  this 

Standard,  Test  Provisions,  should  not  address  off-line  test  (e. 
g.,  ATE,  manual  test  equipment).  It  is  felt  that  this  material 
should  be  part  of  MIL-STD-415. 

MI L-STD-4 15D  —  Design  Criteria  for  Test  Provisions  for 

Electronic  Systems  and  Associated  Equipment. 

This  Standard  depicts  design  criteria  for  test  provisions 
for  electronic  systems  and  associated  equipment.  It  is  outdated. 
The  recommendation  in  the  Rome  Air  Development  Center  report  sug¬ 
gests  that  the  Standard  should  be  updated  to  address  new 
technologies,  such  as  performance  monitoring,  VLSI/VHSIC,  fault 
tolerance,  prognostics,  and  sensor  requirements.  It  is  recom¬ 
mended  that  the  list  of  these  newer  technologies  be  expanded  to 
include  fault  logging  and  reporting  methods,  communication 
standards,  test  of  nonelectronic  equipments,  time-stress  measure¬ 
ment  devices,  etc.  It  is  also  recommended  that  the  document  be 
broadened  to  include  hardware-specific  diagnostics:  for  example, 
test  buses.  It  is  recommended  that  the  revision  to  this  Standard 
be  a  total  rewrite  and  not  merely  a  "band-aid"  revision. 

MIL-STD-2084  (AS)  —  General  Requirements  for  Maintainability 

of  Avionic  and  Electronic  Systems  and 
Equipment. 

This  Standard  has  been  canceled  for  use  on  new  applications. 
The  RADC  report  recommended  extracting  the  valid  portions  of  this 
Standard  and  putting  them  in  various  documents,  including  a  new 
on-line  diagnostic  Standard.  It  is  recommended  that  a  new  on¬ 
line  diagnostic  Standard  not  be  prepared.  Specific,  valid  diag¬ 
nostic  provisions  should  be  placed  in  existing  Standards 
(e.  g.,  MIL-STD-415),  with  title  changes  for  DoD-wide 

appl ication. 
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MIL-STD-1472C  —  Human  Engineering  Design  Criteria  for  Military 

Systems,  Equipment,  and  Facilities. 

The  finding/recommendation  contained  in  the  RADC  report  is 
supported.  It  is  recommended  that  the  Standard  be  analyzed  to 
ascertain  if  test  point  access  is  adequately  covered. 

MIL-STD-2076  (AS)  —  General  Requirements  For  a  Unit  Under  Test 

Compatibility  With  Automatic  Test 
Equipment . 

Agree  with  recommendation  in  the  RADC  report. 

MIL -STD-13 26  (Navy)  —  Test  Point,  Test  Point  Selection,  and 

Interface  Requirements  for  Equipments 
Monitored  by  Shipboard  On-Line  Auto¬ 
matic  Test  Equipment. 

It  is  recommended  that  certain  portions  of  the  Standard, 
such  as  the  disclosure  format,  be  removed  from  this  Standard  and 
placed  in  a  guidance  document.  Examples  of  its  use  with  non¬ 
electronic  systems  should  be  developed,  using  this  disclosure 
format. 

MIL-STD-1345B  (Navy) /MIL-STD-1519  (USAF)  — 

MIL-STD-1519  (Update  for  MATE)  --  Preparation  of  Test  Require¬ 
ments  Documents. 

The  finding/recommendation  and  proposed  activity  in  the  RADC 
report  is  valid,  with  a  proviso  that  the  inclusion  of  automatic 
test  program  generation  and  computer-aided  engineering  and  design 
requirements  in  MIL-STD-1519  be  generic,  to  allow  use  by  many 
different  test  program  generation  systems. 

MIL-STD-2077  (AS)  —  General  Requirements  for  Test  Program 

Sets. 

Appendix  A,  which  addresses  test  program  instruction  formats, 
should  include  means  for  electronic  delivery  of  this  data. 

MIL-STD-1685  (SH)  —  Comprehensibility  Standards  for  Technical 

Manuals  (Metric). 

It  is  recommended  that  this  Standard  address  requirements 
for  electronic  delivery  of  technical  information. 


Requirement  #4:  Conducting  Design  Reviews. 

MIL-STD-1521B  (CJSAF)  —  Technical  Reviews  and  Audits  for 

Systems,  Equipment,  and  Computer 
Programs . 

Agreement  was  reached  on  the  finding/recommendation  con¬ 
tained  in  the  RADC  report,  which  proposed  a  significant  expansion 
for  diagnostic-related  activities  when  conducting  design  reviews. 

Requirement  #5:  Analyzing  and  Assessing  the  Performance  of  the 

Diagnostics  Capability  During  Prime  System/ 
Weapon  System  Design. 

MIL-STD-2080A  (AS)  —  Maintenance  Engineering,  Planning  and 

Analysis  for  Aeronautical  Systems, 
Systems,  Equipment,  and  Support 
Equipment. 

Agreement  was  reached  that  this  Standard  not  be  utilized  by 
the  Air  Force  and  that  it  essentially  duplicates  the  purpose  of 
MIL-STD-1388-1A. 


Requirement  #6; 


Assuring  Delivery  of  Adequate  Diagnostic 


Capability. 


MIL-STD-4  71  — 


Maintainability  Verif ication/Demonstrstion/ 
Evaluation. 


The  following  is  recommended: 


(1)  A  family  of  curves  for  various  confidence  levels  is 
required . 

(2)  The  procedures  in  Appendix  B  identify  a  symbol  that 
represents  sample  size.  However,  nowhere  in  the  proce¬ 
dures  is  the  sample  size  defined.  This  must  be  done  in 
order  to  use  the  procedure. 

(3)  Definition  of  a  sequential  test  or  the  insertion  of 
multiple  faults  must  be  included. 

(4)  One  hundred  percent  diagnostics  prediction/evaluation 
can  be  accomplished  using  the  procedures  contained  in 
MIL-STD-471.  It  is  just  a  matter  of  time  and  money  in 
terms  of  the  iterative  running  of  the  procedures. 
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MIL-STD-781C  —  Reliability  Design  Qualification  and 

Production  Acceptance  Tests:  Exponential 
Distribution. 

The  Standard  should  be  expanded  to  include  Bayesian 
reliability  demonstration  techniques  being  planned  under 
Reliability  Standardization  Project  0026. 

Requirement  #7:  Collecting/Analyzinq  Data  on  the  Performance  of 

the  Diagnostic  Capability. 

MIL-STD-1388-2A  —  Logistic  Support  Analysis:  Data  Element 

Definitions. 

Two  very  similar  routes  of  documenting  failure  mode  data  are 
currently  implemented  in  both  MIL-STD-1629  and  MTL-STD-1388-1A. 
MIL-STD-1629  uses  a  "worksheet"  to  document  failure  modes  and 
MIL-STD-1388-2A  uses  data  sheets  Bl  and  B2  to  document  FMECA . 
Neither  method  provides  additional  information  that  clearly 
separates  one  analysis  from  the  other.  As  a  matter  of  fact, 
1388-2A  states  that  under  no  circumstance  shall  a  contractor  be 
required  to  perform  both  analyses.  Only  one  effort  should  be 
performed. 

Unfortunately,  this  leaves  a  gap  that  needs  to  be  bridged. 
A  means  of  relating  failure  mode  data  to  detection  methods  in  the 
support  of  technical  manual/technical  order  generation  needs  to 
be  emphasized  in  the  LSA  process.  The  current  push  in  the  LSA 
community  is  the  identification  of  failure  modes  that  result  in  a 
direct  relation  to  a  specific  task  (i.  e.,  repair  action, 
inspection,  etc.).  Many  failure  modes,  however,  may  result  in 
the  same  task  and,  therefore,  require  a  lot  of  redundant  effort. 
Consequently,  the  emphasis  in  the  LSA  process  should  be  placed  or* 
the  way  a  failure  is  detected,  not  the  failure  itself.  A 
maintainer/operator  could  care  less  that  a  particular  signal  is 
stuck  high.  He  wants  to  know  the  indication  that  flags  him  to 
start  a  fault  isolation  procedure.  The  result  would  be  an 
analysis  on  how  to  fault  isolate,  not  remove/replace.  This  in¬ 
formation  could  then  be  more  suited  to  the  generation  of  fault 
reporting/isolation  manuals. 
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Requirement  #8:  Standardization  of  Definition 


MIL-STD-721C  --  Definition  of  Terms  for  R&M. 

MIL-STD-1309C  —  Definition  of  Terms  for  TMDE . 

It  is  recommended  that: 

(1)  Diagnostic  terms  should  be  defined  in  a  single 
Standard . 

(2)  Combine  MIL-STD-1309  and  MIL-STD-721C  and  make  the 
title  broader  than  Reliability  and  Maintainability. 
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SECTION  IV 


SUMMARY 


This  report  completes  the  review  of  Military  Standards  by 
the  NS  I A  Integrated  Diagnostics  Group.  It  is  recognized  that  the 
group,  meeting  ii.  a  one-  or  two-day  workshop  environment,  could 
not  provide  detailed  analyses  of  all  of  the  applicable  Standards. 
However,  this  review  provided  a  positive  sounding  board  which  can 
be  used  in  the  revision  of  these  Standards. 
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MISSION 


Rome  Air  Development  Center 


RADC  plans  and  executes  research,  development,  test  and 
selected  acquisition  programs  in  support  of  Command,  Control, 
Communications  and  Intelligence  (C5I)  activities.  Technical  and 
engineering  support  within  areas  of  competence  is  provided  to 
ESD  Program  Offices  (POs)  and  other  ESD  elements  to 
perform  effective  acquisition  of  C3I  systems.  The  areas  of 
technical  competence  include  communications,  command  and 
control,  battle  management  information  processing,  surveillance 
sensors,  intelligence  data  collection  and  handling,  solid  state 
sciences,  electromagnetics,  and  propagation,  and  electronic 
reliability /maintainability  and  compatibility. 


